Research Area:  Machine Learning
Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average 2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.
Keywords:  
Double Deep Q-Learning Model
Energy-Efficient
Edge Scheduling
Machine Learning
Deep Learning
Author(s) Name:  Qingchen Zhang,Man Lin,Laurence T. Yang,Zhikui Chen,Samee U. Khan and Peng Li
Journal name:  IEEE Transactions on Services Computing
Conferrence name:  
Publisher name:  IEEE
DOI:  10.1109/TSC.2018.2867482
Volume Information:  vol. 12,Sept.-Oct. 2019, pp. 739-749
Paper Link:   https://www.computer.org/csdl/journal/sc/2019/05/08449105/13rRUxjQy9m