Abstract

Internet worms are self-replicating malware programs that use the Internet to replicate themselves and propagate to other vulnerable nodes without any user intervention. In addition to consuming the valuable network bandwidth, worms may also cause other harms to the infected nodes and networks. Currently, the economic damage of Internet worms' attacks has reached a level that made early detection and mitigation of Internet worms a top priority for security professionals within enterprise networks and service providers. While the majority of legitimate Internet services rely on the Domain Name System (DNS) to provide the translation between the alphanumeric human memorizable host names and their corresponding IP addresses, scanning worms typically use numeric IP addresses to reach their target victims instead of domain names and hence eliminate the need for DNS queries before new connections are established by the worms. Similarly, modern mass-mailing worms employ their own SMTP engine to bypass local mail servers security measures. However, they still rely on the DNS servers for locating the respective mail servers of their intended victims. Creating host-based Mail eXchange (MX) requests is a violation of the typical communication pattern because these requests are supposed to only take place between mail servers and DNS servers. Several researchers have noted that the correlation of DNS queries with outgoing connections from the network can be utilized for the detection zero-day scanning worms and mass-mailing worms. In this work, we implement an integrated system for the detection and mitigation of zero-day scanning and mass-mailing worms. The detection engine of our system utilizes the above mentioned DNS anomalies of the worm traffic. Once a worm is detected, the firewall rules are automatically updated in order to isolate the infected host. An automatic alert is also sent to the user of the infected host. The system can be configured such that the user response to this alert is used to undo the firewall updates and hence helps reduce the interruption of service resulting from false alarms. The developed system has been tested with real worms in a controlled network environment. The obtained experimental results confirm the soundness and effectiveness of the developed system