Unleash Networks Blog

Network based forensics of P2P traffic has many applications, particularly in law enforcement. Recently, I added preliminary support for Gnutella in Unsniff Network Analyzer as a proof of concept. Wireshark has a rather rudimentary support for this protocol.

We would like to :

1. Monitor what various subjects are searching for on the P2P network
2. Monitor query responses including the filenames and their respective SHA1 hash
3. Identify the file actually downloaded
4. Reconstruct the file actually downloaded

This blog post is a summary of what I found and how you can put this feature to use.

Setup

This was the test setup.

• Limewire 5.5.9 which connects using the Gnutella Protocol 0.6
• Connected to the Gnutella network using the default setttings
• Searched for “flotilla” videos
• Downloaded one of them

All traffic from startup to shutdown of Limewire was captured using Unsniff.  You can convert these files to libpcap format by opening them in Unsniff and selecting File > Export > TCPDUMP

Notes on protocol

A quick note about the P2P protocol elements viewed from a forensics viewpoint.

Bootstrapping

When Limewire first starts up, it uses plain HTTP to download a list of peers from one of many well known hosts. Next it attempts to punch a hole in your firewall using UPnP.  It then selects a suitable peer based on your Geo location and starts a handshake process with it.More details about the bootstrapping process are available at the Gnutella Development Forum. You can observe the bootstrapping process in sessions 1,2,3 in the capture file Limewire_Complete.usnf .

Bootstrap + Punch hole via UPnP + Direct peer handshakes

Use of compression

The handshake takes place over a single TCP connection. The initial part of the handshake is in plaintext, but if both sides signal support for compression, the same TCP connection switches over to stream compression. Unsniff creates a “synthetic” uncompressed TCP stream corresponding to each compressed TCP handshake stream. You can view the uncompressed stream by opening the second capture file Limewire_Handshake.usnf .

f

Compressed original stream + uncompressed synthetic stream created by Unsniff

Use of encryption

Many messages including the actual file transfers are encrypted using AES-128 whose keys are exchanged using Anonymous-Diffie Hellman. This is bad news because our forensics ability effectively ends here. But traffic analysis can give us the some clues about what might have been transferred across the encrypted session.

Actual content encrypted using AES128

Here is the complete stream that was used to transfer the selected file (click for bigger picture). You can view the TLS handshake settling upon the Anon-DH key exchange protocol and AES128 encryption.

Content encypted via TLS : Anon-DH and AES-128

Messages

Gnutella defines a handful of messages and extensions. The relevant protocol messages for this exercise are QUERY and QUERY HIT.  To view these messages,

• switch to the PDU sheet (we are far removed from link layer packets)
• click on a message to view its breakout

QUERY HIT response for "flotilla" videos

Analyzing Gnutella with Unsniff

Viewing Gnutella messages

1. With USNF files, you just have to open the capture file.

2. With PCAP files or live captures, you need to set up Unsniff to recognize Gnutella.

Since Gnutella uses random TCP ports, just assign Port 65535  to Gnutella. This is the special ‘catch all’ port in Unsniff.  All unrecognized TCP traffic will be treated as Gnutella. To do this :  Select Manage > Access Points > Click on TCP > Add Access Point > Enter 65535 + Gnutella.

3. Import the PCAP file or start a live capture

4. Look at the PDU sheet, it contains all the Gnutella messages.

Scripting to automate this task

To really use this in forensics applications you need to be able to script this stuff. Luckily Unsniff’s scripting API allows us to ditch the GUI altogether and run scripts to extract just what we want.

Here is a useful sample script in Ruby.

• This prints a list of  peers, the filenames, and the SHA1 hash.

# ------------------------------------------------------------------
# qhit Print Gnutella Query Hit response details
# ------------------------------------------------------------------
require 'win32ole'
 
USAGE =   "\nUsage   : qhit capture-filename\n"
EXAMPLE = "\nExample : qhit c:\\temp\\limewire-1.usnf\n"
raise USAGE + EXAMPLE unless ARGV.length == 1
 
# Set up input
InputFile = ARGV[0]
UnsniffDB = WIN32OLE.new("Unsniff.Database")
 
# Open capfile and get pdu index
UnsniffDB.OpenForRead(InputFile)
allPDUs = UnsniffDB.PDUIndex
 
p allPDUs.Count
# Scan each packet looking for netflow protocol id
allPDUs.each do |pdu|
	root_field =  pdu.Fields.Item(0)
	qhit_field = root_field.FindField("Query Hit")
 
	unless qhit_field.nil?
	  ip=qhit_field.FindField("IP Address").Value
	  port=qhit_field.FindField("Port").Value
	  hits=qhit_field.FindField("Hits").Value
 
	  print "\nQuery Hit IP #{ip} Port #{port} Hits #{hits}\n"
 
	  qhit_field.SubFields.each do |qhf|
		if qhf.Name == "Query Result"
			file_name = qhf.FindField("File Name").Value
			sha1 = qhf.FindField("Extensions").Value
			print "    #{file_name.ljust(50)} #{sha1[9..40].ljust(32)}\n"
		end
 
	  end
	end
 
end
 
UnsniffDB.Close()

Running this code as

ruby qhit.rb  MyP2PCapture.usnf

Produces this report containing the peer, files available, and their SHA-1 hash.

Query Hit IP 147.253.115.36 Port 4183 Hits 8
    flotilla [BuBanee].mov                             LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    flotilla.mov                                       IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Timbaland - flotilla.mov                           GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    02 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    flotilla [uploaded.by.YlAgEr].torrent              UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla.YlAgEr.zip                                67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 71.196.30.141 Port 30241 Hits 1
    [isoHunt]_2010_-_Super_Mario_Galaxy_2_Soundtrack.5584826.TPB[1].torrent AVB5S3DSP7JX7HWD5SXJFLRD5XIKT44O

Query Hit IP 71.196.30.141 Port 30241 Hits 1
    06 - Bowser Jr.'S Fiery Flotilla.mp3               NS5KQQC642HKYVDQBHG42IE4C3MVSWXZ

Query Hit IP 187.48.197.152 Port 3144 Hits 1
    006 flotilla.wma                                   MRA7J53TK2GOPJ7L2BJZHYZRWANCLVTK

Query Hit IP 127.32.23.144 Port 9707 Hits 1
    flotilla.wma                                       QLX3VNHDFKKCQPLQR2HTA7EZAQV4BJYY

Query Hit IP 166.144.95.59 Port 7018 Hits 8
    flotilla [xxSmokExx].mov                           LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    flotilla.mov                                       IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Kesha - flotilla.mov                               GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    03 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    flotilla.qkelop.torrent                            UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla.qkelop.zip                                67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 166.144.95.59 Port 7018 Hits 8
    flotilla [xxSmokExx].mov                           LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    flotilla.mov                                       IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Kesha - flotilla.mov                               GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    03 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    flotilla.qkelop.torrent                            UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla.qkelop.zip                                67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 48.249.11.129 Port 18420 Hits 8
    flotilla.[SCTV83].mov                              LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    02 flotilla.mov                                    IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Iyaz - flotilla.mov                                GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    06 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    [EZTV].flotilla.[WoDaJe].torrent                   UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla[keygenWoDaJe].zip                         67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 147.253.115.36 Port 4183 Hits 8
    flotilla [BuBanee].mov                             LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    flotilla.mov                                       IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Timbaland - flotilla.mov                           GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    02 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    flotilla [uploaded.by.YlAgEr].torrent              UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla.YlAgEr.zip                                67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 71.196.30.141 Port 30241 Hits 1
    [isoHunt]_2010_-_Super_Mario_Galaxy_2_Soundtrack.5584826.TPB[1].torrent AVB5S3DSP7JX7HWD5SXJFLRD5XIKT44O

Query Hit IP 71.196.30.141 Port 30241 Hits 1
    06 - Bowser Jr.'S Fiery Flotilla.mp3               NS5KQQC642HKYVDQBHG42IE4C3MVSWXZ

Query Hit IP 187.48.197.152 Port 3144 Hits 1
    006 flotilla.wma                                   MRA7J53TK2GOPJ7L2BJZHYZRWANCLVTK

Query Hit IP 127.32.23.144 Port 9707 Hits 1
    flotilla.wma                                       QLX3VNHDFKKCQPLQR2HTA7EZAQV4BJYY

Query Hit IP 166.144.95.59 Port 7018 Hits 8
    flotilla [xxSmokExx].mov                           LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    flotilla.mov                                       IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Kesha - flotilla.mov                               GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    03 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    flotilla.qkelop.torrent                            UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla.qkelop.zip                                67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 166.144.95.59 Port 7018 Hits 8
    flotilla [xxSmokExx].mov                           LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    flotilla.mov                                       IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Kesha - flotilla.mov                               GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    03 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    flotilla.qkelop.torrent                            UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla.qkelop.zip                                67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

Query Hit IP 48.249.11.129 Port 18420 Hits 8
    flotilla.[SCTV83].mov                              LEIC4ZRMAMV6Z6DBUVBYMX2Q34RDVBCU
    02 flotilla.mov                                    IA57KSMRWERTJRCULVFXKUS4H7VHJ7SG
    Iyaz - flotilla.mov                                GVVCPWCN4WHXUFGVJ3PLBI5RY5AEP5S5
    06 flotilla.mov                                    TQNV2GCUQ77RRYYIRWWH7ZTDGWAUQIBR
    [EZTV].flotilla.[WoDaJe].torrent                   UDLWSMLMC45EFYCNTZJT7VMPT7JIJ5ES
    flotilla[keygenWoDaJe].zip                         67JVZU6Z6R6276FP5O572NAELCYVLV4Y
    flotilla [crack][fixed].zip                        WZHQXB62JAVEPF7KLQXYJYH6ZA2DHSFK
    [CRACK].[flotilla].[by FFF].zip                    ZUF3YNZDV5XSCOO7YXXH73SEEYPF4TW4

The SHA1 hash is in Base32.

Conclusions

It is possible to conduct network based forensics on P2P traffic, but it requires tools with advanced reconstruction abilities.

• You CAN conduct surveillance of location of content
• You CAN conduct surveillance of content being searched for
• You CANNOT reconstruct the file actually transferred due to encryption and use of Anon-Diffie-Hellman
• You CAN however get a fair idea, not of forensic evidence quality about what was transferred

—-

Download Unsniff Network Analyzer

Follow me on Twitter

I’d like to illustrate how reassembly/reconstruction can give an investigator the much needed “entry points” into analyzing a dump of packets.

The demo test case is the Forensics Challenge #4 published by the good guys over at Honeynet.org https://www.honeynet.org/challenges/2010_4_voip You have to answer the challenge yourself of course.

1. Just got a dump of packets, now what ?

Most of us would load the thing into Wireshark and see if we can get some clues. While this gives you the best available protocol breakout of every link layer packet, it could be overwhelming. An alternative and potentially more efficient approach is to look at higher layer data first and identity “entry points”. You can then follow these entry points all the way down to bit level protocol decodes. Two such higher layers are content and flows.

2. Content and Flows

Content like images, video, voice calls, HTML pages, are meant for humans while fragmented TCP packets are not. So, a good place for a human analyst to start is to see if the network analysis tool was able to reconstruct content. If you can see the images, web pages viewed, files transferred, hear voice calls, you can get a fair idea of what the dump contains. You should keep in mind the limitations of the tool and the lurking party pooper (encryption).

Another key starting point is flows. It helps you identify the ones with high volumes in a noisy environment. An example : A typical YouTube video view involves dozens of tiny flows and one huge flow that contains the video being viewed.

3. Example : Forensics Challenge #4

Lets import the Forensics Challenge data pcap into Unsniff Network Analyzer.  Unsniff gives you the following levels of reassembly as first class objects (just like packets).

1. Raw link layer packets (like Wireshark)
2. Reassembled PDUs
3. Flows
4. Content (known as User Objects)
5. Basic Statistics

Import the pcap and switch over to the User Objects sheet.

You will see a bunch of HTML/Images/VoIP calls. Clicking on a HTML object renders the page as it was viewed. To reconstruct the HTML you have to enable the option as shown here.

Clicking on HTML renders it - completely offline

Right click the VoIP calls to check if the call is worthy of packet level analysis.

Right click to play each leg

View form being edited

Someone editing form

Conclusion

By starting off at the top layer (content), we know right off the bat that we are dealing with a Trixbox admin session and a VoIP conversation. We know all the pages on the Trixbox server which were visited / edited and the actual VoIP call that took place.  With this context firmly in place we can dive down to the flow level and then to the packet level with a much better understanding of what we are dealing with.

Getting this context would have been time consuming if you started your analysis from the bottom, link layer packets.

——-

You can download and try Unsniff Network Analyzer. It can be your perfect Wireshark complement.

You may also want to try out our upcoming product Trisul Network Metering and Forensics. A Linux based 24×7 surveillance system designed for Network Security Monitoring.

I re-architected Trisul after months of intense coding to be able to take advantage of multiple cores. I just want to share the approach I took for this project.

The options I evaluated were :

• Flow pinning (like in Suricata, the new IDS engine)
  • Packets mapped to hardware thread  based on tuples
• Work stealing
  • Hardware threads if idle, steal stuff to do (see Cilk)

Flow pinning turned in disappointing results largely due to :

• While Trisul does flow tracking and reassembly,  the main chunk of code deals with metering (counting hundreds of data points based on payload content)
• Hard to balance work based only on tuples

Intel’s Threading Building Blocks are the way to go if you want to build on the Cilk style work stealing model. What’s more you get a lot of extra goodies like concurrent containers, atomics, and native threading wrappers.

Armed with TBB, Trisul is completely implemented as a pipeline with a few serial filters and dozens of parallel filters. The advantage of the pipeline pattern is that you get you can run a lot of code on caches that are still “hot“.

The end results are very encouraging.

Here is a screenshot of trisul chewing through the 11GB of packet traces from the LBL-ICSI Enterprise Tracing Project.

340.7% balanced CPU utilization and almost 3.2 times the speed on 1 hardware thread !!

Of late  I have been seeing a good number of Unbrowse SNMP customers using SNMPv3 INFORM messages. This is pleasantly surprising because I had written off this baby as being too complex to setup.

Here is an article explaining how to set up Unbrowse SNMP to receive and respond to SNMPv3 INFORM messages. It covers both the cases of provisioned Engine ID (like Cisco) and Engine ID discovery.

—

The following software updates are also available for download

• Unbrowse SNMP 1.6.1296
• Juniper MIB Packages

So what is Trisul Network Metering and Forensics ?

Here is the Trisul login screen,  I especially liked it because it captured the zen of network security monitoring so effectively.

We observe

• a freeway with traffic in both directions (head lights on right , tail lights on left)
• vehicles move very fast
• its twilight – we can see, kindof

Now imagine, we are asked to keep tabs on what types of cars went by, which semi trucks are suspiciously overloaded, which cars make unusual trips ?  This is roughly what we are asking of network security monitoring.

Trisul approaches NSM from a traffic monitoring centric position. You can contrast that with Sguil that approaches from an alert centric position.

• Trisul contains powerful long term metering and top-n tracking
• Stores full content in a efficient AES128 CTR encrypted ring
• Tracks Flows
• Alerts from 3rd party (accepts Snort input via Unix sockets, working on Unified2 to accept Suricata)
• Rule based full content engine. (eg, track only first MB, exclude subnet, headers only, etc)
• Pull out and save HTTP transactions (IP/TCP reassembly can handle bad frag)

Lets deep dive into Trisul in the next few posts.

Lately, we are working with Netflow quite a bit for our upcoming release of Trisul Network Metering. Here is a tiny script we find invaluable while looking at network captures containing Netflow traffic.

Often while troubleshooting issues we need to look at the raw Netflow records.

For example : You may want to see all the Netflow records sent for IP = 10.22.1.29

Display filters wont get you far because you will still be left with individual packets. These packets themselves can contain dozens of records of which only one or two match.

We turned to Unsniff Scripting and wrote a simple Ruby script that allows you to query for netflow records matching any field value.  We use this script heavily for our internal testing and wish to share it on the blog.

# ------------------------------------------------------------------
# nfquery    Print matching netflow records
# ------------------------------------------------------------------
require 'win32ole'
 
USAGE =   "nfquery <capture-filename> <fieldname>=<value>"
EXAMPLE = "nfquery c:\temp\nfcap.usnf SrcAddress=209.209.1.200"
 
if ARGV.length != 2
 puts USAGE
 exit 1
end
 
# Set up input
InputFile = ARGV[0]
UnsniffDB = WIN32OLE.new("Unsniff.Database")
QueryFieldName = ARGV[1].split('=')[0]
QueryFieldValue = ARGV[1].split('=')[1]
 
# Open capfile and get packet index
 
UnsniffDB.OpenForRead(InputFile)
PacketIndex = UnsniffDB.PacketIndex
 
# Scan each packet looking for netflow protocol id
(0..PacketIndex.Count-1).each do |idx|
 pkt = PacketIndex.Item(idx)
 
 nf_layer = pkt.FindLayerByGUID("{DF2428E1-4843-48CF-B7DD-CCC9E5AE4BC1}")
 next if nf_layer.nil?
 
 pdu_count=nf_layer.FindField("Count").Value.to_i
 (0..pdu_count-1).each do |pduidx|
 nf_pdu = nf_layer.FindField(">NetflowRecord>pdu[#{pduidx}]")
 next if nf_pdu.nil?
 
 target_field=nf_pdu.FindField(QueryFieldName)
 if target_field.Value == QueryFieldValue
 duration_ms = nf_pdu.FindField("End Time").Value.to_i - nf_pdu.FindField("Start Time").Value.to_i
 s_ip = nf_pdu.FindField("SrcAddress").Value
 d_ip = nf_pdu.FindField("DstAddress").Value
 s_port = nf_pdu.FindField("SrcPort").Value
 d_port = nf_pdu.FindField("DstPort").Value
 octets = nf_pdu.FindField("Octets").Value
 proto = nf_pdu.FindField("Protocol").Value
 
 print pkt.ID.to_s.ljust(6) + "  "
 [s_ip,d_ip,s_port,d_port,octets,proto,duration_ms].each do |item|
 print item.to_s.ljust(16) + " "
 end
 print "\n"
 
 end
 end
 
end
 
UnsniffDB.Close()

To use this script.

1. Import the capture file in tcpdump format into Unsniff (or capture off a live interface)

2. Save the file as USNF format

3. Start querying using the script

More :

Unsniff Scripting Home

Download Unsniff

We just updated the Juniper SNMP MIBS to the latest releases.  Sourced from Junipers public website.

The packages available are

• JUNOS_10.0R1.8.zip (5MB)
• ERX_Enterprise_MIBs_10.3.0.zip (7MB)
• ScreenOS-6.3.zip (780KB)

Get them here

Get Unbrowse SNMP here

Install them via Repository > Import Package

We just enhanced the SSL/TLS capabilities of Unsniff Network Analyzer substantially in our newest release (1.8.0.1420)

1. Support for TLS extensions – RFC 4366
2. Support for TLS extensions – RFC 4492 (ec_point_formats and elliptic_curves)
3. Validates if specified key file is in unencrypted PKCS#8 format
4. Support for the latest TLS extension Renegotiation_Info with the tentative extension number of 0xFF01. This is the fix for the TLS MITM Renegotiation Flaw that has been making the rounds the past couple of weeks. See the Internet Draft at http://tools.ietf.org/html/draft-rescorla-tls-renegotiation-00

In this release :

Verify if the specified key file is unencrypted PKCS#8

This is the number one problem people face when using Unsniff for decrypting SSL/TLS. The private key needs to be in unencrypted PKCS#8 format. Prior versions of Unsniff happily allowed you to specify a key in any format, but would log an error message “Invalid Key Material …..” when the time comes to use it.

Any format other than unecrypted PKCS#8 will give this error

TLS Extensions

Here is a screenshot of Unsniff’s support for TLS extensions. Most but not all extensions are completely decoded (not just shown as TLV blobs Type-Length-Value).

TLS extensions completely decoded

But we already have Wireshark

We all use and love Wireshark. But if you work with SSL/TLS a LOT then you need to give Unsniff Network Analyzer a try. It could be useful to have it around in your toolbox along with Wireshark. Specifically, Unsniff could save you bunch of time because (1) it can produce bounce diagrams that you otherwise need to draw by hand (2) it can reassemble upper layer content like web pages (3) it tracks entire SSL records not just ethernet link layer packets (4) scriptable using Ruby (5) share decrypted packet captures without sharing the private keys.

Download Unsniff

The big security news this past week was the SSL Renegotiation Flaw found by researchers Marsh Ray and Steve Dispensa at Phone Factor. Here is what they say :

The SSL authentication gap allows an attacker to mount a man-in-the-middle attack, and affects the majority of SSL-protected servers on the Internet. Specifically, the vulnerability allows the attacker to inject malicious data and commands into the authenticated SSL communications path. This can often be done without either the client or server (e.g. web server and browser) being able to detect the attack.

Read more at their site

Listen to an interview featuring Marsh Ray

What is really cool with this vulnerability is that the researchers have released packet captures of it in action.

Grab the packet captures, a whitepaper, and protocol diagrams from Phone Factor’s site.

Analyzing the packet captures

In this post,let us see how we can analyze these and similar packet captures. There is plenty of heavy lifting to be done, so we will be breaking open a tool called Unsniff. Laura Chappell over at Chappell Seminars has a similar post on analyzing the traces using Wireshark.

Among all the captures released, client_init_renego_bis.pcap seems to demonstrate the attack the best. To make this exercise easy we first decrypt the capture file using Unsniff’s TLS decryption capabilities and save it in Unsniff’s native USNF format.

The USNF format

The USNF format is a great way to share decrypted SSL/TLS packet captures without supplying the private key. No, the private key is not secretly tucked away in the file. It is simply not required after the initial decryption.

Step 1 : Download the latest version of Unsniff

Step 2 : Download the capture file in USNF format from here

Step 3 : Open the file and play with the Packets, PDU, and Sessions tabs.

• Packets : A list of link layer packets like you would see with Wireshark.
• Sessions : A list of TCP streams.
• PDUs : A list of reassembled and decrypted SSL records.

The Packets tab is not very interesting here because SSL records bear no relationship to link layer packet boundaries. So lets focus on the PDU and Sessions tabs.

Analyzing the streams

Switch to the sessions tab and note that the Sessions tab shows 6 TCP streams. But the original pcap file contained only 2 streams (from victim to MITM and from MITM to server). Unsniff creates two extra streams for every SSL/TLS stream. This is an great way to strip and analyze encrypted or tunneled streams.

The two extra streams are :

• The decrypted SSL stream (with all the handshakes in it. Decrypted but with signature and block cipher padding intact)
• The decrypted upper layer HTTP stream (with only the application data)

View HTTP data from the perspective of Server and MITM

Lets first look at the HTTP data from the HTTP server’s perspective.

Move to the Sessions tab and make sure the Stream sheet is activated. You may also have to rt-click and switch to UTF-8.

• Click on Stream 5 ( HTTP data as seen by the server)

What the web server sees (behold the evil)

• Click on Stream 6 (HTTP data sent by the victim)

What the victim sent to the webserver

Going deeper with the Bounce View

SSL Records Bounce Diagram (Click to view generated PDF)

The real juicy part is ofcourse the protocol exchanges between the victim, the MITM, and the server. The ideal tool for this is called the “PDU Bouncer’. Activate it by Right-Clicking on a PDU and selecting Bounce View. This creates a bounce diagram containing all the PDUs that went back and forth on the same TCP stream in time order. For example : Right click on PDU #2 to bring up the diagram shown on the right.

Three-way Bounce

What is really cool about this bounce diagram is that you can select two PDUs from different streams and you will then get a three-way (or even 4 way diagrams).

This is exactly what we need, the victim, the MITM, and the server PDUs all laid out in a protocol diagram. Switch to the PDU view, right click and select PDUs 2 and 14. Now bring up the PDU bouncer.

Lets see what we get.

Stage 1

Shows the MITM (192.168.80.17) intercepting the Client Hello from the victim and then establishing a separate connection to the server with the intention of patching on the client later. The session ID can be seen to be 1C144B..

Showing MITM intercepting victim

Stage 2

This shows the MITM sending the evil payload (PDU#28). Note the evil payload is an incomplete HTTP request sent with the hope that the victim will complete it. Once the payload is sent, you can see a new session being renegotiated. This new session is passed on to the client, from which point on the MITM stays away. The damage having been done ! Pure evil.

Step 2 : Renegotiating a new session and patching the client in

You can generate a PDF for any bounce diagram via Right Click + Print + Select CutePDF.

The PDF for this diagram is available here.

Tailpiece

The authors themselves have a pretty comprehensive whitepaper including a protocol diagram of their own in the package (http://www.phonefactor.com/sslgap/). The purpose of this post is to introduce a new tool you may wish to consider adding to your security toolbox in addition to the ubiquitous and lovable Wireshark.

Unsniff Network Analyzer 2.0 also features remote analysis in line with the NSM principles advocated in the Tao of Network Security Monitoring book. You can download the latest beta builds as they become available from http://www.unleashnetworks.com/downloads. Please support the product by testing out the beta builds.

Who says people who look at raw packets cant have fun ? Let’s see how we can reconstruct YouTube videos from raw packets and play them back directly from Unsniff. We also show you a nifty Ruby script to automate this.

Playback

1. Fire up Unsniff Network Analyzer and start a packet capture

2. Go watch some YouTube video(s)  to completion. We will explain how to deal with half played videos later.

3. Once you are done,  switch over to the User Objects tab where all the action is. You will see a list of “user objects” extracted by Unsniff from the stream. Click on the Type column to sort by Type and locate the FLV objects. Right click to Save or Play (To playback you need the free VLC Player installed and FLV files associated with VLC).

Right click the user object of type FLV and select Play or Save

Feeling adventurous ?  Lets try some more tricks.

Pruning the packet capture

When you interact with a site like YouTube there is a ton of content exchanged, not all of it is the video bits. You have packets containing images, HTML, Flash, and Javascript. The actual FLV is exchanged in a single TCP stream. If you can save these streams alone, you will be able to reconstruct the videos.

Here is how you do it.

1. Switch over to the “Sessions”, here is where you will see a list of TCP sessions updated in real time (every packet).

Only the TCP Stream containing the media is required ! Cut and paste it

2. Locate the TCP Stream carrying FLV traffic. These are typically the really large ones. Select the stream, copy the stream (Edit->Copy) and Paste it as a new file (Edit -> Paste as New). Voila ! Switch to the User Objects in the new file and you will find the FLV without all the other clutter.

Script the whole thing using Ruby

The right clicking novelty quickly wears out after a while. You really want to automate this type of forensic stuff.

Here is a sample which shows you how you can use Ruby to script this stuff.

Task : Extract all the videos in a capture file as separate playable files. (including the ones that were aborted in the middle due to the use losing interest)

1. Save the capture as USNF format (this is the scriptable format used by Unsniff).

#
# extract FLVs from a capture file (USNF format) into a directory
#
require 'win32ole'
 
raise "Usage : xu2be <capture-file>" unless ARGV.length==1  
 
UnsniffDB = WIN32OLE.new("Unsniff.Database")
UnsniffDB.OpenForRead(ARGV[0])
UOIndex = UnsniffDB.UserObjectsIndex
 
(UOIndex.Count-1).times  do |idx|
 uo = UOIndex.Item(idx)
 if  uo.Type == 'FLV'
 uo.SaveToFile("Video_#{idx}.flv")
 print "Saved file Video_#{idx}.flv\n"
 end
end
 
UnsniffDB.Close

2. Run this script like so

C:\vboxshare\CAPS>ruby xu2be.rb incomplete-youtube.usnf
Saved file Video_91.flv
Saved file Video_192.flv
Saved file Video_193.flv

We are still working on tweaking the experience for our upcoming Unsniff 2.0 release. In the meantime, we invite you to  try out the Unsniff Beta 1.8 and give us feedback on what you’d like to really see.