Albuquerque, Rafael Almeida (2024) Machine Learning-based Energy Aware Placement of Container over Virtual Machines. Masters thesis, Concordia University.
Preview |
Text (application/pdf)
5MBAlbuquerque_MCompSc_F2024.pdf - Accepted Version Available under License Spectrum Terms of Access. |
Abstract
The advent of 5G and the imminent arrival of Beyond 5G (B5G) have significantly increased demands on service providers. This exponential growth poses a challenge for 5G networks, since clients are currently offloading data to edge cloud servers to meet the connectivity and latency requirements. These servers must continually scale to meet increasing demands for CPU, memory, and storage, leading to significant energy consumption. Data centers account for 1.5% of global energy consumption and produce equally high greenhouse gas emissions. This trend will grow unless we find ways to improve efficiency. Our work proposes a solution to these problems with an efficient placement algorithm backed by an accurate energy predictive model. This model helps by pre-emptively detecting the energy each machine will consume when future tasks are deployed. These predictive
capabilities help the placement and reduce overall energy consumption. Our model uses Performance Monitoring Counters and various sensors, such as heat and fan speed, commonly found on Data Center machines, to increase its feature space and accuracy. Our work
includes creating the model and integrating it with the energy-aware placement algorithm. Additionally, our method increased the performance and overall Quality of Service. Our results show that our machine learning model, particularly using XGBoost, can reduce energy consumption and improve task completion times in realistic scenarios. Our experiments, tested on real servers with realistic loads, achieved good results without using stresses like stress-ng that generate unrealistic loads. Our model achieved an R2 score of 91.2%, helping reduce energy consumption by 6% without changes to the cluster or the
need for consolidation.
| Divisions: | Concordia University > Gina Cody School of Engineering and Computer Science > Computer Science and Software Engineering |
|---|---|
| Item Type: | Thesis (Masters) |
| Authors: | Albuquerque, Rafael Almeida |
| Institution: | Concordia University |
| Degree Name: | M. Comp. Sc. |
| Program: | Computer Science |
| Date: | 10 July 2024 |
| Thesis Supervisor(s): | Jaumard, Brigitte |
| ID Code: | 994222 |
| Deposited By: | Rafael Almeida Albuquerque |
| Deposited On: | 24 Oct 2024 16:15 |
| Last Modified: | 24 Oct 2024 16:15 |
References:
K. Ahmed, M. Alvarez, and M. Bollen. Characterizing failure and repair time of servers in a hyper-scale data center. In IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe), The Hague, Netherlands, 2020.A. Al-Moalmi, J. Luo, A. Salah, K. Li, and L. Yin. A whale optimization system for energy-efficient container placement in data centers. Expert Systems with Applications, 164:113719, 2021.
J. Baumgartner, C. Lillo, and S. Rumley. Performance losses with virtualization: Comparing bare metal to VMs and containers. In A. Bienz, M. Weiland, M. Baboulin, and C. Kruse, editors, High Performance Computing, Cham, 2023. Springer Nature Switzerland.
A. Beloglazov and R. Buyya. OpenStack Neat: a framework for dynamic and energy-efficient consolidation of virtual machines in OpenStack clouds. Concurrency and Computation: Practice and Experience, 27(5):1310–1333, 2015.
A. Bohra and V. Chaudhary. VMeter: Power modelling for virtualized clouds. In IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), 2010.
L. Caviglione, M. Gaggero, M. Paolucci, and R. Ronco. Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacenters. Soft Computing, 25(19):12569–12588, 2021.
T. Chen and C. Guestrin. Xgboost: A scalable tree boosting system. CoRR, abs/1603.02754, 2016.
C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 20(3):273–297, Sep 1995.
A. H. T. Dias, L. H. A. Correia, and N. Malheiros. A systematic literature review on virtual machine consolidation. ACM Comput. Surv., 54(8), Oct 2021.
X. Fan, W.-D. Weber, and L. A. Barroso. Power provisioning for a warehouse-sized computer. ACM SIGARCH International Symposium on Computer Architecture (ISCA), 35(2):13–23, 2007.
F. Farahnakian, T. Pahikkala, P. Liljeberg, J. Plosila, N. T. Hieu, and H. Tenhunen. Energy-aware VM consolidation in cloud data centers using utilization prediction model. IEEE Transactions on Cloud Computing, 7(2):524–536, 2019.
S. Giallorenzo, J. Mauro, M. G. Poulsen, and F. Siroky. Virtualization costs: benchmarking containers and virtual machines against bare-metal. SN Computer Science, 2(5):404, 2021.
S. Gill, S. Tuli, A. Toosi, F. Cuadrado, P. Garraghan, R. Bahsoon, H. Lutfiyya, R. Sakellariou, O. Rana, S. Dustdar, and R. Buyya. Thermosim: Deep learning based framework for modeling and simulation of thermal-aware resource management for cloud computing environments. Journal of Systems and Software, 166(110596):234–248, April-June 2020.
K. Haghshenas and S. Mohammadi. Prediction-based underutilized and destination host selection approaches for energy-efficient dynamic VM consolidation in data centers. The Journal of Supercomputing, 2020.
L. Helali and M. N. Omri. A survey of data center consolidation in cloud computing systems. Computer Science Review, 39:100366, 2021.
A. Hylick, R. Sohan, A. Rice, and B. Jones. An analysis of hard drive energy consumption. In 2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems, 2008.
S. Ilager and R. Buyya. Energy and thermal-aware resource management of cloud data centres: A taxonomy and future directions. ArXiv, abs/2107.02342, 2021.
C. Jin, X. Bai, C. Yang, W. Mao, and X. Xu. A review of power consumption models of servers in data centers. Applied Energy, 265:114806, 2020.
K. Kaur, S. Garg, G. Kaddoum, F. Gagnon, and D. N. K. Jayakody. Enlob: Energy and load balancing-driven container placement strategy for data centers. In 2019 IEEE Globecom Workshops (GC Wkshps), 2019.
M. A. Khan, A. Paplinski, A. M. Khan, M. Murshed, and R. Buyya. Dynamic Virtual Machine Consolidation Algorithms for Energy-Efficient Cloud Resource Management: A Review, Springer International Publishing, Cham, 2018.
C. Kominos, N. Seyvet, and K. Vandikas. Bare-metal, virtual machines and containers in OpenStack. In Conference on Innovations in Clouds, Internet and Networks (ICIN), 2017.
N. Li, X. Xu, Q. Sun, J. Wu, Q. Zhang, G. Chi, C.-L., and N. Sprecher. Transforming the 5G RAN with innovation: The confluence of cloud native and intelligence. IEEE Access, 11:4443–4454, 2023.
B. Liang, X. Dong, Y. Wang, and X. Zhang. Memory-aware resource management algorithm for low-energy cloud data centers. Future Generation Computer Systems, 113:329–342, 2020.
S. Madan and D. Antoniadis. Leakage current mechanisms in hydrogen-passivated fine-grain polycrystalline silicon on insulator MOSFET’s. IEEE Transactions on Electron Devices, 33(10):1518–1528, 1986.
A. Marathe, Y. Zhang, G. Blanks, N. Kumbhare, G. Abdulla, and B. Rountree. An empirical survey of performance and energy efficiency variation on Intel processors. In Proceedings of the 5th International Workshop on Energy Efficient Supercomputing, E2SC’17, New York, NY, USA, 2017. Association for Computing Machinery.
N. Moocheet, B. Jaumard, P. Thibault, and L. Eleftheriadis. A sensor predictive model for power consumption using machine learning. In IEEE International Conference on Cloud Networking (CLOUDNET), Hoboken, NJ, USA, November 2023.
T. K. Okada, A. De La Fuente Vigliotti, D. M. Batista, and A. Goldman vel Lejbman. Consolidation of VMs to improve energy efficiency in cloud computing environments. In Brazilian Symposium on Computer Networks and Distributed Systems, 2015.
Oracle. Installing and configuring OpenStack (Kilo) in Oracle® Solaris, three-node architecture overview. https://docs.oracle.com/cd/E65465_01/html/E61044/archover.html.
F. Quesnel, H. Mehta, and J.-M. Menaud. Estimating the power consumption of an idle virtual machine. In IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, 2013.
M. Ranjbari and J. Torkestani. A learning automata-based algorithm for energy and SLA efficient consolidation of virtual machines in cloud data centers. Journal of Parallel and Distributed Computing, 113:55–62, 2018.
E. Reshetova, J. Karhunen, T. Nyman, and N. Asokan. Security of OS-level virtualization technologies: Technical report, 2014.
M. Rezaei-Mayahi, M. Rezazad, and H. Sarbazi-Azad. Temperature-aware power consumption modeling in hyperscale cloud data centers. Future Generation Computer Systems, 94:130–139, 2019.
G. Rodola. Psutil documentation, 2020.
L. Stress. stress(1) - linux man page. https://linux.die.net/man/1/stress.
S. Sultan, I. Ahmad, and T. Dimitriou. Container security: Issues, challenges, and the road ahead. IEEE Access, 7:52976–52996, 2019.
D. Thangam et al. Impact of data centers on power consumption, climate change, and sustainability. In K. Kumar, V. Varadarajan, N. Nasser, and R. Poluru, editors, Computational Intelligence for Green Cloud Computing and Digital Waste Management, chapter 4, IGI Global Publishers, USA, March 2024.
B. Wang, Z. Qi, R. Ma, H. Guan, and A. V. Vasilakos. A survey on data center networking for cloud computing. Computer Networks, 91:528–547, 2015.
Y. Wang, D. Nörtershäuser, S. Le Masson, J.-M. Menaud, et al. An empirical study of power characterization approaches for servers. In International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, 2019.
Y. Wang, D. Nörtershäuser, S. Le Masson, J.-M. Menaud, et al. An empirical study of power characterization approaches for servers. In International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, 2019.
C. Zaloumis. Are your data centers keeping you from sustainability?, Feb 2024.
X. Zhang, Z. Shen, B. Xia, Z. Liu, and Y. Li. Estimating power consumption of containers and virtual machines in data centers. In IEEE International Conference on Cluster Computing (CLUSTER), 2020.
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.
Repository Staff Only: item control page
Download Statistics
Download Statistics