Abstract

In the IoT era and with the advent of 5G networks, an enormous amount of data is generated, and new applications require more and more computation power and real-time response. Although cloud computing is a reliable solution to provide computation power, the real-time response is not guaranteed. Thus, the multi-access edge computing (MEC), which consists of distributing the edge servers in the proximity of end-users to have low latency besides the higher processing power, is increasingly becoming a vital factor for the success of modern applications.
Edge server placement and task offloading play a crucial role in the efficient design of MEC architecture. There is a finite discrete set of possible solutions, and finding the optimal one is known to be an NP-hard combinatorial optimization problem. Heuristics, mixed-integer programming, and clustering algorithms are among the most widely used approaches to solve this problem.
Recently, researchers have investigated reinforcement learning (RL) to solve combinatorial optimization problems, which has shown promising results. In this thesis, we propose novel RL-frameworks for solving the joint problem of edge server placement and base station allocation. There are a few studies that have used RL in placement optimization. In our investigation, the focus is on the modeling part to make the Q-learning applicable for a large scale real-world problem.
Therefore, in this research, Q-learning is examined and applied in the edge server placement while considering two significant and striking perspectives. The first one is about minimizing the cost of network design by reducing the delay and the number of edge servers. The second perspective is the placement of K-edge servers to create K-fair-balanced clusters with minimum network delay.
Despite the impressive results of RL, its application in real-world scenarios is highly challenging. Throughout our modeling, the faced issues are explained, and our solutions are provided. Besides, the impact of state representation, action space, and penalty function on the convergence is discussed. Extensive experiments using a real-world dataset from Shanghai demonstrate that in the light of efficient penalty function, the agent is able to find the actions that are the source of higher delayed rewards, and our proposed algorithms outperform the other benchmarks by creating a trade-off among multiple objectives.