Efficient Automated Signatures Extraction Implementation

This work describes a code implementation of a tool for zero day attack signature extraction based on the work “Automated signature extraction for high volume attacks”[1]. The code implementation offers a more correct and faster implementation than the code used to initially verify the work in [1] – the new code implementation offers an increase in throughput and offer more correct signatures.
Given two large sets of messages, P of messages captured in the network at peacetime (i.e., mostly legitimate traffic) and A captured during attack time (i.e., contains many attack messages), the tool extracts a set S of strings, that are frequently found in A and not in P. Therefore, a message containing one of the strings from S is likely to be an attack message. This tool finds popular strings of variable length in a set of messages, using a modified implementation [4] of the Heavy Hitters (Finding Frequent Items) algorithm [3]. This implementation is used as a building block to extract the desired signatures.
Using the attack signatures found by the tool in conjunction with a network traffic- filtering device, a yet unknown attack could be automatically detected and stopped within minutes from attack start time.
The development focused on creating a fast implementation in order to achieve high throughput, which is very important when operating in large traffic networks environment. The development methodology included repeated inspection of code sections, by using CPU/Memory profilers and static code analysis tool. These tools helped finding issues in the code. Specifically, the CPU profiler helped finding code sections with high latency. Once an issue was found it was resolved. A performance evaluation was a major part of the development lifecycle.
The tool is offered as a command line utility and a website was created in order to make it accessible for testing.