Akhter, Urooj Naveed (2025) Performance Evaluation of 5G MIMO DetectorsWith Deep Learning. Masters thesis, Concordia University.
Preview |
Text (application/pdf)
12MBAkhter_MA_F2024 (for Fall).pdf - Accepted Version Available under License Spectrum Terms of Access. |
Abstract
Fifth-generation (5G) wireless networks promise ultra-reliable low-latency communication, enhanced mobile broadband, and massive machine-type connectivity capabilities made possible by advanced technologies such as massive MIMO, OFDM-based modulation, and dynamic spectrum access. However, these benefits introduce new challenges for receiver design, including increased complexity, high data rates, and time-varying channels that degrade the performance of traditional detection methods.
This thesis presents a comprehensive performance analysis of 5G networks using a deep learning–based receiver. A simulation framework is designed to evaluate 5G MIMO receivers under diverse channel environments, including AWGN, Rayleigh fading, and the 3GPP TDL-C model.
A custom dataset is generated using MATLAB’s 5G Toolbox to simulate OFDM-based transmissions across multiple MIMO configurations and modulation schemes. Various neural network architectures
including Fully Connected Neural Network (FCNN), Convolutional Neural Network (CNN), Residual Neural Network (ResNet) and Long Short Term Memory (LSTM network), are implemented and benchmarked against Maximum likelihood (ML), sphere decoding, and MMSE
detection.
Performance metrics including bit error rate (BER), symbol error rate (SER), processing speed, and computational complexity are evaluated. The results highlight trade-offs between accuracy and efficiency and provide insights into the feasibility of deep learning–based detection for practical 5G deployments.
| Divisions: | Concordia University > Gina Cody School of Engineering and Computer Science > Electrical and Computer Engineering |
|---|---|
| Item Type: | Thesis (Masters) |
| Authors: | Akhter, Urooj Naveed |
| Institution: | Concordia University |
| Degree Name: | M.A. Sc. |
| Program: | Electrical and Computer Engineering |
| Date: | 1 June 2025 |
| Thesis Supervisor(s): | Zhu, Wei-Ping and Diniz, Paulo |
| Keywords: | 5G Networks, MIMO transmission, Deep Learning, traditional detectors, CNN, LSTM, ResNet, Sphere Decoding |
| ID Code: | 995695 |
| Deposited By: | Urooj Naveed Akhter |
| Deposited On: | 04 Nov 2025 15:53 |
| Last Modified: | 04 Nov 2025 15:53 |
References:
[1] Kp, “5G NR: Resource grid: https://howltestuffworks.blogspot.com/2019/10/5g-nr-resourcegrid.html.”
[2] M. Hao, H. Li, and G. Xu, “Towards efficient and privacy-preserving federated deep learning,” in 2019 IEEE International Conference on Communications (ICC), Shanghai, China, May 2019, pp. 1–6.
[3] G. S and G. S, “Securing medical image privacy in cloud using deep learning network,” Journal of Cloud Computing, vol. 12, no. 1, p. 40, Mar. 2023.
[4] X. Kui, F. Liu, M. Yang, H. Wang, C. Liu, D. Huang, Q. Li, L. Chen, and B. Zou, “A review of dose prediction methods for tumor radiation therapy,” Meta-Radiology, vol. 2, no. 1, p. 100057, Mar. 2024.
[5] D. Quang, N. Quang, Vo, and C. Nguyen, “BeCaked: An explainable artificial intelligence model for COVID-19 forecasting,” in Proceedings of the ICR’22 International Conference on Innovations in Computing Research, vol. 12, May 2022, pp. 53–64.
[6] M. S. Akbar, Z. Hussain, M. Ikram, Q. Z. Sheng, and S. Mukhopadhyay, “6G survey on challenges, requirements, applications, key enabling technologies, use cases, AI integration issues and security aspects.” arXiv, Oct. 2024.
[7] J. Proakis and M. Salehi, Digital Communications, 5th ed. New York, NY, USA: McGraw Hill, 2007.
[8] B. Hassibi and H. Vikalo, “On the sphere-decoding algorithm I. Expected complexity,” IEEE Transactions on Signal Processing, vol. 53, no. 8, pp. 2806–2818, Aug. 2005.
[9] M. Damen, H. El Gamal, and G. Caire, “On maximum-likelihood detection and the search for the closest lattice point,” IEEE Transactions on Information Theory, vol. 49, no. 10, pp. 2389–2402, Oct. 2003.
[10] Z. Guo and P. Nilsson, “Algorithm and implementation of the k-best sphere decoding for MIMO detection,” IEEE Journal on Selected Areas in Communications, vol. 24, no. 3, pp. 491–503, Mar. 2006.
[11] S. R. Doha and A. Abdelhadi, “Deep learning in wireless communication receiver: A survey,” in ArXiv, vol. abs/2501.17184, doha, Qatar, Jan. 2025.
[12] N. Samuel, T. Diskin, and A. Wiesel, “Learning to detect,” IEEE Transactions on Signal Processing, vol. 67, no. 10, pp. 2554–2564, May 2019.
[13] T. J. O’Shea and J. Hoydis, “An introduction to deep learning for the physical layer,” IEEE Transactions on Cognitive Communications and Networking, vol. 3, no. 4, pp. 563–575, Dec. 2017.
[14] K. Noor, A. L. Imoize, C.-T. Li, and C.-Y. Weng, “A review of machine learning and transfer learning strategies for intrusion detection systems in 5G and beyond,” Mathematics, vol. 13, no. 7, p. 1088, 2025.
[15] G. Foschini and M. Gans, “On limits of wireless communications in a fading environment when using multiple antennas,” Wireless Personal Communications, vol. 6, no. 3, pp. 311–335, Mar. 1998.
[16] E. Telatar, “Capacity of multi-antenna gaussian channels,” European Transactions on Telecommunications, vol. 10, no. 6, pp. 585–595, Nov. 1999.
[17] A. Paulraj, R. Nabar, and D. Gore, Introduction to Space-Time Wireless Communications. United States: Cambridge University Press, Jun. 2008.
[18] Z. Guo and P. Nilsson, “Reduced complexity schnorr-euchner decoding algorithms for MIMO systems,” IEEE Communications Letters, vol. 8, no. 5, pp. 286–288, May 2004.
[19] E. Viterbo and J. Boutros, “A universal lattice code decoder for fading channels,” IEEE Transactions on Information Theory, vol. 45, no. 5, pp. 1639–1642, Jul. 1999.
[20] M. Biguesh and A. B. Gershman, “Training-based MIMO channel estimation: A study of estimator tradeoffs and optimal training signals,” IEEE Transactions on Signal Processing, vol. 54, Mar. 2006.
[21] B. Marinberg, A. Cohen, E. Ben-Dror, and H. Permuter, “A study on MIMO channel estimation by 2D and 3D convolutional neural networks,” in 2020 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS). New Delhi, India: IEEE,
2020, pp. 1–6.
[22] W. Xia, G. Zheng, Y. Zhu, J. Zhang, J.Wang, and A. P. Petropulu, “Deep learning based beamforming neural networks in downlink MISO systems,” in 2019 IEEE International Conference on Communications Workshops (ICC Workshops), May 2019, pp. 1–5.
[23] F. A. Aoudia and J. Hoydis, “End-to-end learning for OFDM: From neural receivers to pilotless communication | IEEE journals & magazine | IEEE xplore,” in IEEE Transactions on Wireless Communications, vol. 21, Feb. 2022.
[24] T. H. Ahmed, J. J. Tiang, A. Mahmud, C. Gwo Chin, and D.-T. Do, “Deep reinforcement learning-based adaptive beam tracking and resource allocation in 6G vehicular networks with switched beam antennas,” Electronics, vol. 12, no. 10, p. 2294, Jan. 2023.
[25] N. Farsad, H. B. Yilmaz, A. Eckford, C.-B. Chae, andW. Guo, “A comprehensive survey of recent advancements in molecular communication,” IEEE Communications Surveys & Tutorials,
vol. 18, no. 3, pp. 1887–1919, 2016.
[26] J. R. Hershey, J. L. Roux, and F.Weninger, “Deep unfolding: Model-based inspiration of novel deep architectures,” in MITSUBISHI ELECTRIC RESEARCH LABORATORIES. arXiv, Sep. 2014.
[27] H. Ye, G. Y. Li, and B. H. Juang, “Power of deep learning for channel estimation and signal detection in OFDM systems,” IEEE Wireless Communications Letters, vol. 7, no. 1, pp. 114–117, Feb. 2018.
[28] P. Wang, A. Jiang, X. Liu, J. Shang, and L. Zhang, “LSTM-based EEG classification in motor imagery tasks,” IEEE transactions on neural systems and rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society, vol. 26, no. 11, pp.
2086–2095, Nov. 2018.
[29] Y. Luo, Z. Chen, and T. Yoshioka, “Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,” in ICASSP 2020. Barcelona, Spain.: IEEE, Mar. 2020, pp. 46–50.
[30] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: arXiv, Dec. 2015, pp. 770–778.
[31] Burera, S. Ahmed, and S. Kim, “Transformer learning-based efficient MIMO detection method,” Physical Communication, vol. 70, p. 102637, Jun. 2025.
[32] A. Fuchs, C. Knoll, N. N. Moghadam, A. P. J. Huang, E. Leitinger, and F. Pernkopf, “Selfattention for enhanced OAMP detection in MIMO systems,” in Self-Attention for Enhanced OAMP Detection in MIMO Systems. Rhodes Island, Greece: arXiv, 2023, pp. 1–5.
[33] J. Lee, “Set transformer: A framework for attention-based permutation-invariant neural networks,” in ICML 2019, Long Beach, California.
[34] Z.-Q. Wang, S. Cornell, S. Choi, Y. Lee, B.-Y. Kim, and S. Watanabe, “TF-GridNet: Making time-frequency domain models great again for monaural speaker separation,” in ICASSP 2023. Rhodes Island, Greece: IEEE, Mar. 2023, pp. 1–5.
[35] L. Yang, W. Liu, and W. Wang, “TFPSNet: Time-frequency domain path scanning network for speech separation,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Singapore, Singapore: IEEE, 2022, pp. 6842–6846.
[36] A. Kumar, S. Chakravarty, S. Suganya, H. Sharma, R. Pareek, M. Masud, and S. Aljahdali, “Intelligent conventional and proposed hybrid 5G detection techniques,” Alexandria Engineering Journal, vol. 61, no. 12, pp. 10 485–10 494, Dec. 2022.
[37] D. Griffin and J. Lim, “Signal estimation from modified short-time fourier transform,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 32, no. 2, pp. 236–243, Apr. 1984.
[38] A. Gu, K. Goel, and C. R´e, “Efficiently modeling long sequences with structured state spaces,” Aug. 2022.
[39] Y. Ran-Milo, E. Lumbroso, E. Cohen-Karlik, R. Giryes, A. Globerson, and N. Cohen, “Provable benefits of complex parameterizations for structured state space models,” in NeurIPS 2024, vol. abs/2410.14067. arXiv, Oct. 2024, p. 12.
[40] S. Zhang, M. Zhang, X. Pang, and M. Yang, “No-skim: Towards efficiency robustness evaluation on skimming-based language models,” https://arxiv.org/html/2312.09494v2, 2023.
[41] N. Nayak, T. Tholeti, M. Srinivasan, and S. Kalyani, “Green DetNet: Computation and memory efficient DetNet using smart compression and training,” arXiv preprint arXiv:2003.09446, vol. abs/2003.09446, Apr. 2021.
[42] M. Goutay, F. A. Aoudia, and J. Hoydis, “Deep HyperNetwork-based MIMO detection,” in Deep HyperNetwork-Based MIMO Detection. Atlanta, GA, USA: arXiv, 2020, pp. 1–5.
[43] S. Han, H. Mao, andW. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” in International Conference on Learning Representations. arXiv, 2016.
[44] A. Papoulis and S. Pillai, Probability, Random Variables and Stochastic Processes, 4th ed. Boston, USA: McGraw-Hill Education, 2002.
[45] L. Zheng and D. Tse, “Diversity and multiplexing: A fundamental tradeoff in multiple-antenna channels,” IEEE Transactions on Information Theory, vol. 49, no. 5, pp. 1073–1096, May 2003.
[46] 3GPP, “3rd generation partnership project,” 3GPP, Technical Specification Group Radio Access Network TR 25.913 V7.3.0 (2006-03), 2006.
[47] A. A. Zaidi, R. Baldemair, H. Tullberg, H. Bjorkegren, L. Sundstrom, J. Medbo, C. Kilinc, and I. Da Silva, “Waveform and numerology to support 5G services and requirements,” IEEE Communications Magazine, vol. 54, no. 11, pp. 90–98, Nov. 2016.
[48] F. Rinaldi, A. Raschell`a, and S. Pizzi, “5G NR system design: A concise survey of key features and capabilities,” Wireless Networks, vol. 27, no. 8, pp. 5173–5188, Nov. 2021.
[49] D. Tse and P. Viswanath, Fundamentals of Wireless Communication. NY, United States: Cambridge University Press, Jul. 2005.
[50] S. M. Kay, Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory. River, NJ, United States: Prentice-Hall, Inc., Mar. 1993, vol. 1.
[51] M. K. Simon and M.-S. Alouini, “Digital communications over fading channels (M.K. simon and M.S. alouini; 2005) [book review],” IEEE Transactions on Information Theory, vol. 54, no. 7, pp. 3369–3370, Jul. 2008.
[52] T. S. Rappaport, Wireless Communications: Principles and Practice. Upper Saddle River, NJ: Prentice Hall, 1996.
[53] S. Verdu, Multiuser Detection. Princeton University, New Jersey, USA: Cambridge Univeristy Press, Aug. 1998.
[54] B. Hochwald and S. ten Brink, “Achieving near-capacity on a multiple-antenna channel,” IEEE Transactions on Communications, vol. 51, no. 3, pp. 389–399, Mar. 2003.
[55] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
[56] F. A. Aoudia and J. Hoydis, “Model-free training of end-to-end communication systems,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 11, pp. 2503–2516, Nov. 2019.
[57] A. Alkhateeb, “DeepMIMO: A generic deep learning dataset for millimeter wave and massive MIMO applications,” in Proc. of Information Theory and Applications Workshop (ITA). arXiv, Feb. 2019.
[58] A. Klautau, P. Batista, N. Gonzalez-Prelcic, Y. Wang, and R. W. H. Jr, “5G MIMO data for machine learning: Application to beam-selection using deep learning,” in 2018 Information
Theory and Applications Workshop (ITA). San Diego, CA, USA: arXiv, Jun. 2021, pp. 1–9.
[59] J. Hoydis, S. Cammerer, F. A. Aoudia, A. Vem, N. Binder, G. Marcus, and A. Keller, “Sionna: An open-source library for next-generation physical layer research,” in arXiv:2203.11854, vol. abs/2203.11854. arXiv, 2022.
[60] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in The 3rd International
Conference for Learning Representations, vol. 9. San Diego: arXiv, 2015.
[61] A. Goldsmith, Wireless Communications. Stanford University, California: Cambridge University Press, 2005.
Repository Staff Only: item control page


Download Statistics
Download Statistics