Abstract

IoT systems are increasingly deployed across various domains, making end-to-end (E2E) testing crucial to ensuring expected functionality. However, testing IoT systems is challenging due to their heterogeneity, distributed execution, and real-world constraints. Traditional testing approaches are inefficient and limited, focusing on isolated layers rather than complete system interactions.
This thesis presents findings from a Systematic Literature Review (SLR) and an Industry Study on IoT system testing, analyzing existing challenges, approaches, and tools. Based on these insights, we propose a taxonomy for testing IoT systems and introduce a framework for evaluating the technical software engineering (SE) testable aspects of IoT systems. Building on this foundation, we propose an approach for functional end-to-end (E2E) testing of IoT systems, leveraging Use Case Specifications (UCSs) written in a restricted format and IoT systems descriptions. Our approach systematically converts UCSs into executable test scenarios, which are further transformed into structured test data (i.e., payload). This payload provides the necessary data for generating test cases that cover multiple layers of the IoT system. The generated test cases are then executed on the system under test (SUT) to detect bugs. To evaluate the proposed approach, we conducted an empirical study on an IoT system, analyzing its effectiveness in test case generation and bug detection. The results demonstrate that our approach detects bugs across multiple layers of the IoT system by leveraging the use of real-time execution data. Furthermore, it significantly improves test coverage and efficiency, reducing manual effort while maintaining accuracy. These findings indicate that the use of real-time execution data at each layer enhances bug detection in IoT systems.