Figure 11 - uploaded by Young Hyun Cho
Content may be subject to copyright.
Source publication
Damage caused by the recent series of application-level network attacks clearly indicate an immediate need for in- creased security. Most of these attacks can be more accu- rately detected by a technique termed Deep Packet Inspec- tion. Deep packet inspection not only examines the packet header, but also looks through the entire payload search- ing...
Citations
... Nevertheless, DPI system can be shaped to discover "CodeRed" worm by looking for a string pattern ".ida?" in the packet's payload [5] [6]. It appears unavoidable that next-generation firewalls will require some kind of DPI to maximize network security [7]. ...
... For example, if we have n elements to be inserted to the Bloom filter where n=1000, and if we suppose that the false positive rate is p=0.01, the size of BF m = −n * ln(p) ln2 2 . Then the value of k will be (7). Whereas, in FTDF we need to employ a single hash function without any concern to the values of (n) and (m). ...
: Deep Packet Inspection (DPI) represents the major process in network intrusion detection and prevention systems. In
DPI each security threat is represented as a signature, and the payload of every incoming data packet is matched against the
set of current signatures. Moreover, DPI is also used for other networking applications like packet classification, QoS techniques,
protocol identification and so on. DPI exhausts extra CPU and memory resources, and as a result, several attempts have been
proposed to improve this process. In this paper, we proposed a fast two dimensional filter (FTDF) with low false positive rate for
DPI purposes. It consists of the two-dimensional array that employs single hash function and has very low false positive rate. Using this filter as an identification tool in a DPI technique will result in more accurate and higher throughput than other systems that employ Bloom and Quotient filters. Our experiments show that the proposed solution has time improvement up to 94% over others that employ Bloom or Quotient filters and the achieved average throughput is 1.8 Gbps.
... However, they have generally suffered from limited hardware resources, especially the tight budget of memory resources to store patterns or perform pattern comparison. Cho et al. [6] proposed to relieve the scalability issue of pattern comparison and to improve the performance by using 8-to-1 decoders for each byte comparator and instantiating one decoder output for all the same output. The 8-to-1 decoder was setting the character decoder to the first stage of the pipeline. ...
The performance gap between the execution speed of security software and the amount of data to be processed is ever widening. A common solution is to close the performance gap through hardware implementation of security functions. However, continuously expanding signature databases have become a major impediment to achieving scalable hardware based pattern matching. Additionally, evolutionary rule databases have necessitated real time online updating for reconfigurable hardware implementations. Based on the observation that signature patterns are constructed from combinations of a limited number of primary patterns, we propose to decompose the Snort signature patterns. These smaller primary pattern sets can be stored along with their associations to allow dynamic signature pattern reconstruction. Not only does the matching operation potentially become more scalable, but the real time online updating task is simplified. The approach is verified with patterns from the latest version of the Snort rule database. The experimental results show that after decomposition, a reduction in size of over 77% can be achieved on Snort signature patterns.
... Recently, specialized hardware [68,21] for intrusion detection in high-volume networks has been developed. These solutions have used Field-programmable gate array (FPGA) to implement the intrusion detection systems. ...
The rapid growth in malicious Internet activity, due to the rise of threats like
automated worms, viruses, and botnets, has driven the development of tools
designed to protect host and network resources. One approach that has gained
significant popularity is the use of network based security
systems. These systems are deployed on the network to detect, characterize and
mitigate both new and existing threats.
Unfortunately, these systems are developed and deployed in production networks
as generic systems and little thought has been paid to customization.
Even when it is possible to customize these devices, the approaches for
customization are largely manual or ad hoc. Our observation of the production
networks suggest that these networks have significant diversity in end-host
characteristics, threat landscape, and traffic behavior -- a collection of
features that we call the security context of a network. The scale and
diversity in security context of production networks make manual or ad hoc
customization of security systems difficult. Our thesis is that automated
adaptation to the security context can be used to significantly improve the
performance and accuracy of network-based security systems.
In order to evaluate our thesis, we explore a system from three broad categories
of network-based security systems: known threat detection, new threat detection,
and reputation-based mitigation. For known threat detection, we examine a
signature-based intrusion detection system and show that the system performance
improves significantly if it is aware of the signature set and the traffic
characteristics of the network. Second, we explore a large collection of
honeypots (or honeynet) that are used to detect new threats. We show that
operating system and application configurations in the network impact honeynet
accuracy and adapting to the surrounding network provides a significantly better
view of the network threats. Last, we apply our context-aware approach to a
reputation-based system for spam blacklist generation and show how traffic
characteristics on the network can be used to significantly improve its
accuracy.
We conclude with the lessons learned from our experiences adapting to network
security context and the future directions for adapting network-based security
systems to the security context.
... Matching large sets of patterns against an incoming stream of data is a fundamental task in several fields such as network security [1][2][3][4][5][6][7][8][9][10][11][12] or computational biology [13,14]. For example, high-speed network Intrusion Detection Systems (IDS) rely on efficient pattern matching techniques to analyze the packet payload and make decisions on the significance of the packet body. ...
... All authors are members of the European Network of Excellence on High-Performance Embedded Architecture and Compilation (HiPEAC) pattern matching most of them in the area of reconfigurable hardware [1][2][3][4][5][6][7][8][9][10][11][12]18]. In general FPGAs are well suited for this task, since designs can be customized for a particular set of search patterns and updates to that set can be performed via reconfiguration. ...
... More recently, several hashing techniques were proposed for IDS pattern matching. Cho and Mangione-Smith proposed the use of prefix matching to read the remaining pattern from a memory and reduce the area requirements [7,11]. However, this approach has limited performance and the restriction of patterns having short unique prefixes. ...
In this paper, we consider hardware-based scanning and analyzing packets payload in order to detect hazardous contents. We present two pattern matching techniques to compare incoming packets against intrusion detection search patterns. The first approach, decoded partial CAM (DpCAM), predecodes incoming characters, aligns the decoded data, and performs logical and on them to produce the match signal for each pattern. The second approach, perfect hashing memory (PHmem), uses perfect hashing to determine a unique memory location that contains the search pattern and a comparison between incoming data and memory output to determine the match. Both techniques are well suited for reconfigurable logic and match about 2200 intrusion detection patterns using a single Virtex2 field-programmable gate-array device. We show that DpCAM achieves a throughput between 2 and 8 Gb/s requiring 0.58-2.57 logic cells per search character. On the other hand, PHmem designs can support 2-5.7 Gb/s using a few tens of block RAMs (630-1404 kb) and only 0.28-0.65 logic cells per character. We evaluate both approaches in terms of performance and area cost and analyze their efficiency, scalability, and tradeoffs. Finally, we show that our designs achieve at least 30% higher efficiency compared to previous work, measured in throughput per area required per search character.
... For a Spartan3-1000 the design would probably use even more logic cells, since the ISE tool uses more resources (when available) to increase performance. 5 Cho et al. report LUTs instead of slices. If they had used slices, their designs would have higher area cost and lower PEM. ...
Pattern matching is one of the most computationally intensive tasks in network security systems. Numerous pattern matching approaches have been proposed in the past. The most common ones use: regular expres-sions, discrete comparators or CAM, Pre-decoding, and Hashing to match patterns. The researchers' first concern was to achieve high operating throughput in order to process incoming packets in wire rates. Since the set of matching patterns increases rapidly, though, pattern matching designers started considering also the area cost of their designs. In this paper, we at-tempt an evaluation of FPGA-based pattern matching techniques for network security systems. We measure the efficiency of pattern matching modules in terms of obtained performance per area cost.
... This is achieved by partitioning the rules in an optimized evaluation structure. Recently, specialized hardware [15, 16] for intrusion detection in high-volume networks has been developed. However, hardware-based solutions are complex to modify (e.g., to change the detection algorithm). ...
Intrusion detection and prevention systems have become es- sential to the protection of critical networks across the Internet. Widely deployed IDS and IPS systems are based around a database of known malicious signatures. This database is growing quickly while at the same time the signatures are getting more complex. These trends place ad- ditional performance requirements on the rule-matching engine inside IDSs and IPSs, which check each signature against an incoming packet. Existing approaches to signature evaluation apply statically-defined opti- mizations that do not take into account the network in which the IDS or IPS is deployed or the characteristics of the signature database. We argue that for higher performance, IDS and IPS systems should adapt accord- ing to the workload, which includes the set of input signatures and the network trac characteristics. To demonstrate this idea, we have devel- oped an adaptive algorithm that systematically profiles attack signatures and network trac to generate a high performance and memory-ecient packet inspection strategy. We have implemented our idea by building two distinct components over Snort: a profiler that analyzes the input rules and the observed network trac to produce a packet inspection strategy, and an evaluation engine that pre-processes rules according to the strategy and evaluates incoming packets to determine the set of ap- plicable signatures. We have conducted an extensive evaluation of our workload-aware Snort implementation on a collection of publicly avail- able datasets and on live trac from a border router at a large university network. Our evaluation shows that the workload-aware implementation outperforms Snort in the number of packets processed per second by a factor of up to 1.6x for all Snort rules and 2.7x for web-based rules with reduction in memory requirements. Similar comparison with Bro shows that the workload-aware implementation outperforms Bro by more than six times in most cases.
... Most pattern matching hardware based on FPGA [11, 12, 13, 14, 15, 16, 17] likely fails to satisfy the worst-case performance requirement. When the number of patterns is increased, the operating frequency of FPGA pattern matching hardware tends to increase due to the increase in the amount of combinational circuits for state transitions. ...
... To the authors' best knowledge, this is the first work that successfully addresses these two requirements. Two papers [16, 17] addressed issues related to the pattern match information requirement and proposed two similar architectures that generate signature indexes using pruned priority binary tree and highly pipelined binary-OR tree. However, these architectures cannot handle multiple matches that simultaneously occur and also do not provide any information on the location of a match in a payload. ...
Pattern matching is one of critical parts of Network Intrusion Prevention Systems (NIPS). Pattern matching hardware for NIPS should find a matching pattern at wire speed. However, that alone is not good enough. First, pattern matching hardware should be able to generate sufficient pattern match information including the pattern index number and the location of the match found at wire speed. Second, it should support pattern grouping to reduce unnecessary pattern matches. Third, it should show constant worst-case performance even if the number of patterns is increased. Finally it should be able to update patterns in a few minutes or seconds without stopping its operations. We modify Shift-OR hardware accelerator and propose a system architectures to meet the above requirement. Using Xilinx FPGA simulation, we show the new system scaled well to achieve a high speed over 10Gbps and satisfies all of the above requirements.
... If we assume only one token is detected at any given clock cycle, the address encoder can be built using the combination of outputs from the binary tree of OR gates [7]. ...
In this paper, we present reconfigurable hardware architecture for detecting semantics of streaming data on 1+ Gbps networks. The design leverages on the characteristics of context-free-grammar (CFG) that allows the computers to understand the semantics of data. Although our parser is not a true CFG parser, we use the linguistic structure defined in the grammars to explore a new way of parsing data using Field Programmable Gate Array (FPGA) hardware. Our system consists of pattern matchers and a syntax detector. The pattern matchers are automatically generated using the grammar token list while the syntax detector is generated based on the aspects of the grammar that define the order of all possible token sequences. Since all the rules are mapped onto the hardware as parallel processing engines, the meaning of each token can be determined by monitoring where it is being processed. Our highly parallel and fine grain pipelined engines can operate at a frequency above 500 MHz. Our initial implementation is XML content-based router for XML remote procedure calls (RPC). The implementation can process the data at 1.57 Gbps on Xilinx VirtexE FPGA and 4.26 Gbps on the Virtex 4 FPGA.
... However, due to high processing requirement, implementing such a detector using a general purpose processor is costly for 1+ gigabit network. Therefore, many researchers have developed several cost-efficient high-performance pattern matching engines and processors for deep packet inspection [7,12,13,4,19,2,8,16,6,5]. ...
... The researchers have also shown that the area efficient pattern detectors can be built by optimizing the group of byte comparators that are pipelined according the patterns [7,18,19,2,8]. Our earlier works use chains of byte comparators and read-only-memory (ROM) to reduce the amount of logic by storing parts of the data in memory [3,4,6]. ...
Due to increasing economic damage from computer network intrusions, many routers have built-in firewalls that can classify packets based on header information. Such classification engine can be effective in stopping attacks that target protocol specific vulnerabilities. However, they are not able to detect worms that are encapsulated in the packet payload. One method used to detect such application-level attack is deep packet inspection. Unlike the most firewalls, a system with a deep packet inspection engine can search for one or more specific patterns in all parts of the packets. Although deep packet inspection increases the packet filtering effectiveness and accuracy, most of the current implementations do not extend beyond recognizing a set of predefined regular expressions. In this paper, we present an advanced inspection engine architecture that is capable of recognizing language structures described by context-free grammars. We begin by modifying a known regular expression engine to function as the lexical analyzer. Then we build an efficient multi-threaded parsing co-processor that processes the tokens from the lexical analyzer according to the grammar.
... The logic savings is achieved by using the decoders to generate the address for the partial pattern entry in a ROM. By balancing the use of the discrete gates and memory, this yields the highest performance per gate, thus far [3,13]. ...
It has been estimated that computer network worms and virus caused the loss of over $55B in 2003. Network security system use techniques such as deep packet inspection to detect the harmful packets. While software intrusion detection system running on general purpose processors can be updated in response to new attacks. They lack the processing power to monitor gigabit networks. We present a high performance pattern matching co-processor architecture that can be used to monitor and identify a large number of intrusion signature. The design consists of a bank of pattern matchers that are used to implement a highly concurrent filter. The pattern matchers can be programmed to match multiple patterns of various lengths, and are able to leverage the existing databases of threat signatures. We have been able to program the filters to match all the payload patterns defined in the widely used Snort network intrusion detection system at a rate above 7 Gbps, with memory space left to accommodate threat signatures that become available in the future.
