New self monitoring performance stats in Trisul Network Analytics

The latest release of Trisul Network Analytics [ release details ]  has a really nifty self-monitoring feature. There is no need for outside tools like atop or nmon to check on Trisul’s performance any more.

To use the feature :

  • Login as admin
  • Select Tools > Perf Stats

The Perf stats dashboard is shown similar to the one below.

Self Performance Stats Module in Trisul
Self Performance Stats Module in Trisul

The modules shown in the Perf Stats dashboard are

Global Flush Time

Trisul is a streaming network analytics engine. Since there is a continuous stream of high volume data, one key requirement is that aggregated results are flushed out within a small amount of time.  Global Flush-0, 1 are the two default backend threads of Trisul flushing out in parallel. You can have upto 8 backend threads in that case all 8 will be shown on the graph.

Key thing to look for : See if there is spikiness in the chart, all data should ideally be less than 20 seconds.

CPU Usage

Shows Total and Trisul CPU Usage over time.

Lookout for : CPU being pegged. Ideally Trisul CPU usage must rise and fall with load.  Unusual spikes in low traffic times have led us to find scans and DoS attacks as the Trisul TCP reassembly engine is stressed during these times.


Memory Usage

Total and Trisul Memory Usage

Lookout for : Memory climbing , this is very very rare and could indicate a memory leak.   Trisul has a lot of hi-water and lo-water marks to restrict memory usage. If your server has a lot of memory you could consider increasing these numbers so Trisul can make more judicious use of RAM.


Disk Usage

Total and Trisul Usage.  Shows the amount of disk used by the Trisul backend database.

Lookout for : If data climbs steadily and threatens to it the maximum, you may need to adjust the SlicePolicy to reduce the number of slices kept in the system. This way the DB size is kept smaller than the available disk space.


With this new feature, your Trisul monitors itself.


Have you tried Trisul Network Analytics yet ? It is a free download and can immediately start providing value to you !

How to carve out files from network traffic captures for malware analysis

I stumbled across this post on “” blog about the recent compromise titled “Extracting files from network traffic capture“. In that blog, the author has demonstrated file carving using Wireshark and other tools.

There is also a link to a Barracuda PCAP file (1.3MB) contains some malware

I’d like to introduce you to Unsniff Network Analyzer‘s nifty file extraction that addresses the following issues in the PCAP.

  • The EXEs are transferred as content type “text/html”
  • All files have to be written to disk before you can do a file * and pick out the EXEs

The latest version of Unsniff has two extremely useful features that can really speed up this process. Each User Object now has two new attributes

  • Magic String :  We take the first 4 bytes of each content and create a human readable string
  • MD5 Hash : Each user object has a MD5 content hash

These are computed online as traffic is being processed. Once they are stored in an Unsniff Capture File Format (*.USNF) you can just access them instantaneously without reprocessing.

Magic number – pick out EXE transferred as text/html

As mentioned, the Barracuda PCAP drops EXE malware as text/html. In the screenshot below, If you notice User Object 11 – the Type column shows “HTML” but the Magic column shows “MZ90.00.”  Thats a dead giveaway that the content isnt really HTML. Next, you can click on the corresponding MD5 column to start checking with VirusTotal etc.

Files like CSS/JS/HTML usually just have the first 4 bytes of text as the magic number. You can simply ignore them. The best part is this feature works for all files transferred – whether as EMAIL attachment, as FTP files, as Chat file transfers, etc, etc.

Magic number and MD5 Hash shown inline
Magic number and MD5 Hash shown inline


Saving all the Malware EXE files

The way you save the malware files  is to simply “Ctrl-Click” and select all those who start with the EXE Magic number. Then Right Click and Save.

Select by Type or Magic number MZ.. indicates Windows EXE
Select by Type or Magic number MZ.. indicates Windows EXE


What gives Unsniff even more power is that the entire process above is scriptable in Ruby or VBScript. You never have to open a single GUI window. You can use the Unsniff Scripting API to automatically chew through PCAPS and continuously dump only the EXE files this way.

If you are into info sec, Unsniff Network Analyzer  is a tool that you really need to have in your kit. Download for free today.


Fun with Unsniff scripting : save last few packets of each flow

There was a question on the Wireshark Q&A Site. A user wanted to apply a filter to only show the last few packets of all TCP flows. This can be done manually in Wireshark quite easily but the user had hundreds of flows and was looking for an automatic way to do this.

Here is a quick post with code that demonstrates how you can automate this and other custom analysis using Unsniff Network Analyzer.

Unsniff exposes an object model to scriptland. This means that flows (a.k.a streams), packets, user objects, PDUs are all top level objects. All you have to do is to grab the flows collection, then for each flow save the last 6 packets. Here is the script.

' lastn - Dump last 5 packets of every stream into a separate PCAP file

Set Sout = WScript.StdOut

If WScript.Arguments.Count <> 2 Then
  Sout.WriteLine "Usage: cscript last5.vbs input-pcap output-pcap "
End If 

InputTCPD = WScript.Arguments.Item(0)
OutputTCPD = WScript.Arguments.Item(1)

Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")

' Import from input libpcap to USNF format
Set UnsniffDB = CreateObject("Unsniff.Database")
UnsniffDB.New("temp_cap.usnf" )
UnsniffDB.Import "libpcap", InputTCPD
Sout.WriteLine "Imported tcpdump file " & InputTCPD

' New USNF - to be exported to output file
Set UnsniffDBOut = CreateObject("Unsniff.Database")
UnsniffDBOut.New("temp_cap2.usnf" )
Sout.WriteLine "Opened usnf output file "

Dim STIndex
Set STIndex = UnsniffDB.StreamIndex
For Each ST In STIndex
  With ST
  Set Pkts = ST.Packets
  If Pkts.Count > 5 Then
    FromIdx = Pkts.Count-6
    ToIdx = Pkts.Count-1
    FromIdx = 0
    ToIdx = Pkts.Count-1
  End If

  Sout.WriteLine "Exporting packets " & FromIdx & " to " & ToIdx & " in stream " & ST.ID

  For Idx = FromIdx To ToIdx

  End With

UnsniffDBOut.Export "libpcap",OutputTCPD

fso.DeleteFile "temp_cap.usnf"
fso.DeleteFile "temp_cap2.usnf"

The main part of the code is

  • Get the StreamIndex
  • Iterate over all streams; for each stream; grab the last 5 packets
  • Add those packets to the new output file
Set STIndex = UnsniffDB.StreamIndex
 For Each ST In STIndex
 Set Pkts = ST.Packets

To run this script ;

Run as

cscript lastn.vbs inputfile.pcap outputfile.pcap

What you will end up with is a new capture file containing only the last 5 segments of each flow. See pictures below.

Output PCAP file only has last 6 segments per flow
Output PCAP file only has last 6 segments per flow

The automation capabilities of Unsniff can save precious time. Please visit the Unsniff Scripting Guide for more.