一种基于深度强化学习的资源调度方法
A Resource Scheduling Method Based on Deep Reinforcement Learning
DOI: 10.12677/CSA.2021.117205, PDF,    国家科技经费支持
作者: 赵飞鸿, 孙立峰：清华大学计算机科学与技术系，北京
关键词: 云计算；资源调度；深度强化学习；Cloud Computing； Resource Scheduling； Deep Reinforcement Learning
摘要: 启发式调度算法作为云数据中心资源调度的常用方法能够给出一种可行的资源调度策略，但针对特定集群环境设计合适的启发式算法需要较专业的先验知识，而使用通用的启发式方法会对集群资源造成极大浪费。对Kubernetes集群资源调度过程进行建模，设计基于深度强化学习的资源调度方法，对于不同的集群环境和优化目标，使用学习的方法从历史数据中学习得到相应的调度策略，优化集群资源利用情况。实验结果表明，使用该方法比现有启发式方法在任务带权周转时间上平均减少7.5%，最高减少13.5%。
Abstract: As a common method of resource scheduling in cloud data centers, heuristic scheduling algorithm can give a feasible resource scheduling strategy. However, designing a suitable heuristic algorithm for a specific cluster environment requires professional prior knowledge, or using general heuristic methods will cause a great waste of cluster resources. We design a resource scheduling method on Kubernetes system based on deep reinforcement learning. For different cluster environments and optimization goals, using learning methods to obtain corresponding scheduling strategies from historical data can optimize cluster resource utilization. Experimental results show that the use of this method reduces the slowdown of tasks by an average of 7.5% and the highest reduction of 13.5% compared with the existing heuristic methods.
文章引用：赵飞鸿, 孙立峰. 一种基于深度强化学习的资源调度方法[J]. 计算机科学与应用, 2021, 11(7): 2008-2018. https://doi.org/10.12677/CSA.2021.117205

参考文献

[1] 凤凰网科技. 字节跳动在美租用数据中心: 数十万台服务器, 能耗达53兆瓦[EB/OL].
https://tech.ifeng.com/c/80zDZm54HDc, 2020-10-30.
[2] 叶宇飞. 基于深度强化学习的数据中心资源算法调度研究[D]: [硕士学位论文]. 成都: 电子科技大学, 2019.
[3] Ferguson, A.D., Bodik, P., Kandula, S., et al. (2012) Jockey: Guaranteed Job Latency in Data Parallel Clusters. Proceedings of the 7th ACM European Conference on Com-puter Systems, Bern, 10-13 April 2012, 99-112. [Google Scholar] [CrossRef] 
[4] Martinez, J.F. and Ipek, E. (2009) Dynamic Multicore Resource Management: A Machine Learning Approach. IEEE Micro, 29, 8-17. [Google Scholar] [CrossRef] 
[5] Mao, H., Alizadeh, M., Menache, I., et al. (2016) Resource Management with Deep Reinforcement Learning. Proceedings of the 15th ACM Workshop on Hot Topics in Networks, Atlanta, GA, 9-10 November 2016, 50-56. [Google Scholar] [CrossRef] 
[6] Mao, H., Schwarzkopf, M., Venkatakrishnan, S.B., et al. (2019) Learning Scheduling Algorithms for Data Processing Clusters. Proceedings of the ACM Special Interest Group on Data Communication, Beijing, 19-23 August 2019, 270-288. [Google Scholar] [CrossRef] 
[7] LeCun, Y., Bottou, L., Bengio, Y., et al. (1998) Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86, 2278-2324. [Google Scholar] [CrossRef] 
[8] Liu, N., Li, Z., Xu, J., et al. (2017) A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning. 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, 5-8 June 2017, 372-382. [Google Scholar] [CrossRef] 
[9] Mnih, V., Badia, A.P., Mirza, M., et al. (2016) Asynchronous Methods for Deep Reinforcement Learning. International Conference on Machine Learning, PMLR, New York, 20-22 Jun 2016, 1928-1937.
[10] Schulman, J., Wolski, F., Dhariwal, P., et al. (2017) Proximal Policy Optimization Algo-rithms. arXiv:1707.06347 [cs.LG]
[11] Ross, S., Gordon, G. and Bagnell, D. (2011) A Reduction of Imitation Learning and Structured Prediction To No-Regret Online Learning. Proceedings of the Fourteenth International Conference on Artifi-cial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, Fort Lauderdale, 11-13 April 2011, 627-635.
[12] Grandl, R., Ananthanarayanan, G., Kandula, S., et al. (2014) Multi-Resource Packing for Cluster Sched-ulers. ACM SIGCOMM Computer Communication Review, 44, 455-466. [Google Scholar] [CrossRef]