Prototype Intelligent Log-Based Intrusion Detection System

The maintenance of web server security is a daunting task today. Threats arise from hardware failures, software flaws, tentative probing and worst of all malicious attacks. Analysing server logs to detect suspicious activities is regarded as a key form of defence, however, their sheer size makes human log analysis challenging. Additionally, traditional intrusion detection systems rely on methods based on pattern-matching techniques which are not sustainable given the high rates at which new attack techniques are launched every day. The aim of this paper is to develop a proto-type intelligent log based intrusion detection system that can detect known and unknown intrusions automatically. Under a data mining framework, the intrusion detection system is trained with unsupervised learning algorithms specifically the k-means algorithm and the One Class SVM (Support Vector Machine) algorithm. The development of the prototype system is limited to machine generated logs due to lack of real access log files. However, the system's development and implementation proved to be up to 85% accurate in detecting anomalous log patterns within the test logs.