Taxonomy of malicious URL detection techniques

Abstract

Malicious URLs are often used by phishing campaigns, botnets and other attacks. Indeed, DNS traffic is necessary for the Internet to function correctly, which means that this data flow cannot be blocked. For these reasons, detecting malicious URLs is both important, challenging and still an open research problem. There are two types of techniques used to detect malicious URLs: rules-based and machine-learning based. The traditional, rules-based techniques rely on blacklists and heuristics. These techniques struggle to keep up with a rapidly changing array of malicious URLs. Therefore, machine learning-based techniques have emerged. These techniques rely on URL characteristics such as length, number of vowels and others to classify them as legitimate or malicious. The main contribution of this paper is to propose a taxonomy of detection techniques and to point out which URL characteristics are used by each method. While surveys on the topics exist, a precise mapping between the detection methods and the characteristics is not available and we propose one. We also compare these techniques, highlighting that machine learning-based techniques are more complex to implement but better at keeping up with rapidly incoming new malicious URLs. In contrast, rules-based techniques are simpler and easier to implement, but they struggle to update fast enough to identify new malicious URLs.