• A comparison of reinforcement learning algorithms in fairness-oriented OFDMA schedulers

      Comșa, Ioan-Sorin; Zhang, Sijing; Aydin, Mehmet Emin; Kuonen, Pierre; Trestian, Ramona; Ghinea, Gheorghiţă; Brunel University; University of Bedfordshire; University of the West of England; HEIA-FR, Switzerland; et al. (MDPI, 2019-10-14)
      Due to large-scale control problems in 5G access networks, the complexity of radio resource management is expected to increase significantly. Reinforcement learning is seen as a promising solution that can enable intelligent decision-making and reduce the complexity of different optimization problems for radio resource management. The packet scheduler is an important entity of radio resource management that allocates users’ data packets in the frequency domain according to the implemented scheduling rule. In this context, by making use of reinforcement learning, we could actually determine, in each state, the most suitable scheduling rule to be employed that could improve the quality of service provisioning. In this paper, we propose a reinforcement learning-based framework to solve scheduling problems with the main focus on meeting the user fairness requirements. This framework makes use of feed forward neural networks to map momentary states to proper parameterization decisions for the proportional fair scheduler. The simulation results show that our reinforcement learning framework outperforms the conventional adaptive schedulers oriented on fairness objective. Discussions are also raised to determine the best reinforcement learning algorithm to be implemented in the proposed framework based on various scheduler settings. View Full-Text
    • Enhancing user fairness in OFDMA radio access networks through machine learning

      Comşa, Ioan-Sorin; Zhang, Sijing; Aydin, Mehmet Emin; Kuonen, Pierre; Trestian, Ramona; Ghinea, Gheorghiţă; Brunel University; University of Bedfordshire; University of the West of England; HEIA-FR; et al. (IEEE, 2019-06-13)
      The problem of radio resource scheduling subject to fairness satisfaction is very challenging even in future radio access networks. Standard fairness criteria aim to find the best trade-off between overall throughput maximization and user fairness satisfaction under various types of network conditions. However, at the Radio Resource Management (RRM) level, the existing schedulers are rather static being unable to react according to the momentary networking conditions so that the user fairness measure is maximized all time. This paper proposes a dynamic scheduler framework able to parameterize the proportional fair scheduling rule at each Transmission Time Interval (TTI) to improve the user fairness. To deal with the framework complexity, the parameterization decisions are approximated by using the neural networks as non-linear functions. The actor-critic Reinforcement Learning (RL) algorithm is used to learn the best set of non-linear functions that approximate the best fairness parameters to be applied in each momentary state. Simulations results reveal that the proposed framework outperforms the existing fairness adaptation techniques as well as other types of RL-based schedulers.
    • Towards 5G: a reinforcement learning-based scheduling solution for data traffic management

      Comşa, Ioan-Sorin; Zhang, Sijing; Aydin, Mehmet Emin; Kuonen, Pierre; Lu, Yao; Trestian, Ramona; Ghinea, Gheorghiţă; Brunel University; University of Bedfordshire; University of the West of England; et al. (IEEE, 2018-08-06)
      Dominated by delay-sensitive and massive data applications, radio resource management in 5G access networks is expected to satisfy very stringent delay and packet loss requirements. In this context, the packet scheduler plays a central role by allocating user data packets in the frequency domain at each predefined time interval. Standard scheduling rules are known limited in satisfying higher Quality of Service (QoS) demands when facing unpredictable network conditions and dynamic traffic circumstances. This paper proposes an innovative scheduling framework able to select different scheduling rules according to instantaneous scheduler states in order to minimize the packet delays and packet drop rates for strict QoS requirements applications. To deal with real-time scheduling, the Reinforcement Learning (RL) principles are used to map the scheduling rules to each state and to learn when to apply each. Additionally, neural networks are used as function approximation to cope with the RL complexity and very large representations of the scheduler state space. Simulation results demonstrate that the proposed framework outperforms the conventional scheduling strategies in terms of delay and packet drop rate requirements.