About Vivek Rajagopalan

Vivek Rajagopalan is the lead developer behind the Unsniff Network Analyzer, Trisul, and Unbrowse SNMP products.

How to carve out files from network traffic captures for malware analysis

I stumbled across this post on “behindthefirewalls.com” blog about the recent PHP.com compromise titled “Extracting files from network traffic capture“. In that blog, the author has demonstrated file carving using Wireshark and other tools.

There is also a link to a Barracuda PCAP file (1.3MB) contains some malware http://barracudalabs.com/downloads/5f810408ddbbd6d349b4be4766f41a37.pcap

I’d like to introduce you to Unsniff Network Analyzer‘s nifty file extraction that addresses the following issues in the PCAP.

  • The EXEs are transferred as content type “text/html”
  • All files have to be written to disk before you can do a file * and pick out the EXEs

The latest version of Unsniff has two extremely useful features that can really speed up this process. Each User Object now has two new attributes

  • Magic String :  We take the first 4 bytes of each content and create a human readable string
  • MD5 Hash : Each user object has a MD5 content hash

These are computed online as traffic is being processed. Once they are stored in an Unsniff Capture File Format (*.USNF) you can just access them instantaneously without reprocessing.

Magic number – pick out EXE transferred as text/html

As mentioned, the Barracuda PCAP drops EXE malware as text/html. In the screenshot below, If you notice User Object 11 – the Type column shows “HTML” but the Magic column shows “MZ90.00.”  Thats a dead giveaway that the content isnt really HTML. Next, you can click on the corresponding MD5 column to start checking with VirusTotal etc.

Files like CSS/JS/HTML usually just have the first 4 bytes of text as the magic number. You can simply ignore them. The best part is this feature works for all files transferred – whether as EMAIL attachment, as FTP files, as Chat file transfers, etc, etc.

Magic number and MD5 Hash shown inline

Magic number and MD5 Hash shown inline

 

Saving all the Malware EXE files

The way you save the malware files  is to simply “Ctrl-Click” and select all those who start with the EXE Magic number. Then Right Click and Save.

Select by Type or Magic number MZ.. indicates Windows EXE

Select by Type or Magic number MZ.. indicates Windows EXE

 

What gives Unsniff even more power is that the entire process above is scriptable in Ruby or VBScript. You never have to open a single GUI window. You can use the Unsniff Scripting API to automatically chew through PCAPS and continuously dump only the EXE files this way.

If you are into info sec, Unsniff Network Analyzer  is a tool that you really need to have in your kit. Download for free today.

 

Fun with Unsniff scripting : save last few packets of each flow

There was a question on the Wireshark Q&A Site. A user wanted to apply a filter to only show the last few packets of all TCP flows. This can be done manually in Wireshark quite easily but the user had hundreds of flows and was looking for an automatic way to do this.

Here is a quick post with code that demonstrates how you can automate this and other custom analysis using Unsniff Network Analyzer.

Unsniff exposes an object model to scriptland. This means that flows (a.k.a streams), packets, user objects, PDUs are all top level objects. All you have to do is to grab the flows collection, then for each flow save the last 6 packets. Here is the script.

'
' lastn - Dump last 5 packets of every stream into a separate PCAP file

Set Sout = WScript.StdOut

If WScript.Arguments.Count <> 2 Then
  Sout.WriteLine "Usage: cscript last5.vbs input-pcap output-pcap "
  WScript.Quit
End If 

InputTCPD = WScript.Arguments.Item(0)
OutputTCPD = WScript.Arguments.Item(1)

Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")

' Import from input libpcap to USNF format
Set UnsniffDB = CreateObject("Unsniff.Database")
UnsniffDB.New("temp_cap.usnf" )
UnsniffDB.Import "libpcap", InputTCPD
Sout.WriteLine "Imported tcpdump file " & InputTCPD

' New USNF - to be exported to output file
Set UnsniffDBOut = CreateObject("Unsniff.Database")
UnsniffDBOut.New("temp_cap2.usnf" )
Sout.WriteLine "Opened usnf output file "

Dim STIndex
Set STIndex = UnsniffDB.StreamIndex
For Each ST In STIndex
  With ST
  Set Pkts = ST.Packets
  If Pkts.Count > 5 Then
    FromIdx = Pkts.Count-6
    ToIdx = Pkts.Count-1
  Else
    FromIdx = 0
    ToIdx = Pkts.Count-1
  End If

  Sout.WriteLine "Exporting packets " & FromIdx & " to " & ToIdx & " in stream " & ST.ID

  For Idx = FromIdx To ToIdx
    UnsniffDBOut.AddPacket(Pkts.Item(Idx))
  Next

  End With
Next

UnsniffDB.Close()
UnsniffDBOut.Export "libpcap",OutputTCPD
UnsniffDBOut.Close()

fso.DeleteFile "temp_cap.usnf"
fso.DeleteFile "temp_cap2.usnf"

The main part of the code is

  • Get the StreamIndex
  • Iterate over all streams; for each stream; grab the last 5 packets
  • Add those packets to the new output file
Set STIndex = UnsniffDB.StreamIndex
 For Each ST In STIndex
 Set Pkts = ST.Packets

To run this script ;

Run as

cscript lastn.vbs inputfile.pcap outputfile.pcap

What you will end up with is a new capture file containing only the last 5 segments of each flow. See pictures below.

Output PCAP file only has last 6 segments per flow

Output PCAP file only has last 6 segments per flow

The automation capabilities of Unsniff can save precious time. Please visit the Unsniff Scripting Guide for more.

Nifty new context aware packet filters in Unsniff Network Analyzer

We observed a very common pattern among users of Unsniff and well as Wireshark. A majority of the time display filters were being used on a combination of the 5-tuples of a TCP or a UDP packet. This was over 90% of the cases. We decided to work on a nifty solution.

A context aware packet filter.

  • Right click on any packet to bring up a “Pull out packets” menu with automatic filter choices.
  • Select a choice to pull out only matching packets into a new capture file window.
  • Unsniff also has “invert” choices. Use them if you want to only select packets NOT matching the selected criteria.

Now, you can zip through capture files effortlessly and narrow down your analysis data.

 

Start saving time and energy while perusing packet captures. Download the latest version of Unsniff Network Analyzer

 

Some screenshots follow.

Automatic filters based on currently selected packet

and the inverted filter

Pull out packets NOT matching these criteria into a new capture window

More exciting news about Unsniff Network Analyzer coming soon.