Anomaly-based IDS will monitor network traffic and compare it against an established baseline. The baseline will identify what is “normal” for that network- what sort of bandwidth is generally used, what protocols are used, what ports and devices generally connect to each other- and alert the administrator or user when traffic is detected which is anomalous, or significantly different, than the baseline.
A signature based IDS will monitor packets on the network and compare them against a database of signatures or attributes from known malicious threats. This is similar to the way most antivirus software detects malware. The issue is that there will be a lag between a new threat being discovered in the wild and the signature for detecting that threat being applied to your IDS. During that lag time your IDS would be unable to detect the new threat.
While anomaly-based IDS systems might detect an unknown attack, most signature-based IDS will miss a new exploit if there is no rule written for it. IDS systems must receive vendor signature updates. Even if updates are applied, exploits that are unknown to the IDS vendor will not be caught by the signature-based system. Attackers may also try to evade the IDS by using their techniques, exploits or tools. These evasive techniques include flooding, fragmentation, encryption, and obfuscation.
- Flooding- IDSs depend on resources such as memory and processor power to effectively capture packets, analyze traffic, and report malicious attacks. By flooding a network with noise traffic, an attacker can cause the IDS to exhaust its resources examining harmless traffic. In the meantime, while the IDS is distracted and occupied by the volume of noise traffic, the attacker can target its system with little or no intervention from the IDS.
- Fragmentation-Because different network media allow variable maximum transmission units (MTUs), you must allow for the fragmentation of these transmission units into differently sized packets or cells. Hackers can take advantage of this fragmentation by dividing attacking packets into smaller and smaller portions that evade the IDS but cause an attack when reassembled by a target host.
- Encryption-Network-based intrusion detection (covered later in this chapter) relies on the analysis of traffic that is captured as it traverses the network from a source to its destination. If a hacker can establish an encrypted session with its target host using Secure Shell (SSH), Secure Socket Layer (SSL), or a virtual private network (VPN) tunnel, the IDS cannot analyze the packets and the malicious traffic will be allowed to pass. Obviously, this technique requires that the attacker establish a secure encrypted session with its target host.
- Obfuscation-Obfuscation, an increasingly popular evasive technique, involves concealing an attack with special characters. It can use control characters such as the space, tab, backspace, and Delete. Also, the technique might represent characters in hex format to elude the IDS. Using Unicode representation, where each character has a unique value regardless of the platform, program, or language, is also an effective way to evade IDSs. For example, an attacker might evade an IDS by using the Unicode character c1 to represent a slash for a Web page request.