In order to improve the resource allocation ability of optical networks for massive and differentiated power services and reduce the algorithm training time of large-scale services, a smart grid optical network resource allocation scheme based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm was proposed. The large-scale and differentiated power services were considered, the optical core network slice model of smart grid was built and the optimization problem aiming at maximizing the income of power grid companies was proposed. Conditional judgment mapping was proposed to simplify the optimization problem. At the same time, the improved MADDPG algorithm was designed to reduce the training time and meet the real-time needs of the network by placing different services to different agents. Lastly, simulation results show that the proposed algorithm has better reward, lower cost and time delay, and lesser training time.