DNS tunneling detection through statistical fingerprints of protocol messages and machine learning

The use of covert-channel methods to bypass security policies has increased considerably in the recent years. Malicious users neutralize security restriction by encapsulating protocols like peer-to-peer, chat or http proxy into other allowed protocols like Domain Name Server (DNS) or HTTP. This paper illustrates a machine learning approach to detect one particular covert-channel technique: DNS tunneling.Despite packet inspection may guarantee reliable intrusion detection in this context, it may suffer of scalability performance when a large set of sockets should be monitored in real time. Detecting the presence of DNS intruders by an aggregation-based monitoring is of main interest as it avoids packet inspection, thus preserving privacy and scalability. The proposed monitoring mechanism looks at simple statistical properties of protocol messages, such as statistics of packets inter-arrival times and of packets sizes. The analysis is complicated by two drawbacks: silent intruders (generating small statistical variations of legitimate traffic) and quick statistical fingerprints generation (to obtain a detection tool really applicable in the field).Results from experiments conducted on a live network are obtained by replicating individual detections over successive samples over time and by making a global decision through a majority voting scheme. The technique overcomes traditional classifier limitations. An insightful analysis of the performance leads to discover a unique intrusion detection tool, applicable in the presence of different tunneled applications. © 2014 John Wiley & Sons, Ltd.