Profiling packet processing on multicore systems

Unleash Networks is a member of the Intel Software Partner Program. Thanks to the program we have access to a powerful tool for measuring multi core performance of the new Trisul Open Source Network Metering and Forensics tool. I installed the 45-day eval of the Intel Thread Profiler for Windows with the Linux Data Collector.
I then measured Trisul‘s packet processing performance on a dual core system. The results are at the Project Wiki Site.

1mcore.jpg

From my various experiments with threading packet processing, I am leaning towards these conclusions
1. Getting packet processing right on multiple cores is hard.

2. Traditional threading systems seem to be very difficult to get right. Given the volume of tokens (packets), it is easy to incur too much synchronization overhead or severely impact cache performance.

3. Task based approaches like the Intel Threading Building Blocks appear more attractive.

The next major task would be to create a quick prototype application using the Intel TBB library and revisit the measurements.

Trisul news :

I got some email pointing out that the DEB and RPM packages were missing. Sorry, they will be up shortly. The packager is broken.
There are many packet processing tools like Ntop, Snort, Sancp, Argus, etc. Trisul will hopefully find a niche because of its ability to reduce traffic data to a SQL database and its extensible architecture that allows other functions to be plugged in. (Documentation about the architecture is not yet available but see the sysplugs directory in the source code)

Trisul – Sourceforge and Google Code setup

fetch.png

Project Hosting

Trisul is a new open source project that is targeted at security analysts. I set up both Google Code and Sourceforge project sites. As much as I like SF, its performance leaves a lot to be desired. It also loads a lot of external content which adds to its load time. I will use the issue tracker and download link on Google Code. Perhaps as the project matures, we can revisit Sourceforge.

Blog

I also created a wordpress blog called trisul.wordpress.com

Domain

I purchased the domain trisul.org. Eventually, the project will move there. We probably need a VPS if we want to host a demo of Web Trisul (the Ruby on Rails web frontend to the network metering data)

Todo List

Just playing with some options here. I quickly checked out tadalist and todoist. I could not find an easy way to publicly share list items on todoist, so I chose tadalist. The public tasks page is here

New code

The first release on sourceforge (0.4.116) was an embarassing mess. This was due to my unfamiliarity with autoconf and friends. I had just zipped up the source directory as a tarball instead of “make distcheck”. The new release takes care of that.

————————————————–

Some questions people ask me about Trisul. I will try to answer them in the next blog post.

1) Is this project too ambitious ? Can one system integrate traffic monitoring, raw data recording, session tracking, and forensics ?

2) When good stuff like SANCP, Time Machine, ntop, argus, are already available – what beverage is Trisul bringing to the party ?

3) Trisul is at best a single “sensor” or “observation point”, how does it plan to integrate into a centralized console like SGUIL etc.

 

Announcing Trisul

We are happy to announce a major new open source project called Trisul.

fetch.png

What is Trisul ?

Trisul is a network metering and forensics tool. You can install Trisul on any Linux box and have it look at network traffic in real time or via capture files. It meters the traffic (by host, by protocol, by subnet, etc) and stores the results in a SQL database. Trisul also includes a Ruby on Rails application called Web Trisul that allows you to use a web browser to view data in the form of pretty charts.

Status

Trisul has been in development for a few months now primarily as a remote probe for the upcoming Unsniff 2.0 release. We decided to make it open source once we cleaned up some embarassing bits of code. The entire software is GPLv3.

You can install Trisul right away and do some really useful stuff with it. It is still rough around the edges in terms of documentation and the occasional stability problem.

Get it !

We encourage all network administrators especially those involved in security operations to try out Trisul. 

The Trisul Project Site

Trisul Sourceforge Download Page

Â