A Hybrid Approach for Intrusion Detection System

Buffer Overflow (BOF) have become the most common target for network-based attacks and on the other side many detection and prevention techniques have been developed to secure the systems and networks known Intrusion Detection Systems (IDS). The paper deals with the problem of BOF and proposes an IDS which is a combination of Host Intrusion Detection System (HIDS) and Network Intrusion Detection System (NIDS). It is designed to detect any attempt of BOF attack that use the Call/Jump Register technique depending on the use of set of available memory addresses of Call/Jump instructions for loaded DLL files uses them as a return addresses that point to the attacker malicious code being used to exploit the system. The proposed system generates two signature files, one for HIDS and the other for NIDS. The Monitoring and Detection Engine in the HIDS depend on On-Access-Scan technique to capture any file that contains the attack signature as they open and log them to a log file. Besides that, the Monitoring and Detection Engine in the NIDS depends on Snort system to sniff and capture any data packets in the network traffic that contain the attack signature and log them to another log file. An Analysis Engine applies a set of statistical operations and a Fuzzy Analysis System on log files in order to produce a set of reports in the form of PHP web sites that represent the analysis output that give the degree of BOF attack risk.