Hi Vivek,

apologies for a delayed response.

I've applied the BPF but unfortunately I'm still getting the cryptic segfault:

I'm not 100% that IPv6 traffic is being filtered though.

Under Customize > Capture Adapter > Edit Capture Profile,
I tried adding not ip6 and with the speechmarks "not ip6" (just in case that'd make a difference), however after restarting the Trisul daemon, if I go Tools > Live Meters > HostsIPv6, I can still live data for IPv6 traffic? Both global and link local sessions. Would you expect this page to be blank if IPv6 was being filtered?

Many Thanks

Chris,

Live meters are only current to about 3-5 mins, if the segfault happened before that, it would continue to show the last valid data which includes the IP6 addresses.

Do you happen to have various TCP reassembly options enabled ? Check these two in trisulConfig.xml (in /usr/local/etc/trisul )

set them to FALSE and see if it makes a difference. They can use quite a bit of memory.

If it still crashes, we can also try to get the libdnswatcher.so out of the way again. I know we did a lot of hardening in 2.4 but it is possible the problem still lurks.


Can you also see if dmesg has anything to say ?

If none of the above work, I will send you a special build with debug symbols which will pin point the error. Besides that we have dramatically reduced CPU usage in the past 3-4 days in 2.4.

Thanks a lot !

I disabled the suggested options but was still getting a segfault error. I ran dmesg and saw the following:


root@trisul:~# sudo /usr/local/bin/trisul -nodemon /usr/local/etc/trisul/trisulConfig.xml -mode onlinerxring
Segmentation fault


root@trisul:~# dmesg | grep trisul
[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.32-33-generic-pae root=/dev/mapper/trisul-root ro quiet
[ 4.715517] Adding 2813944k swap on /dev/mapper/trisul-swap_1. Priority:-1 extents:1 across:2813944k
[49367.110630] trisul[1490]: segfault at 14 ip 081a0fc0 sp ae0fef50 error 4 in trisul[8048000+323000]


I then ran the cleanenv command and starting Trisul again, since then it's been running all day without error.

Could it be that although we were filtering out IPv6, existing IPv6 traffic logged in the database could cause a problem?

I'll see how it fares and try re-enabling the disabled config options tomorrow, one at a time, to see if the segfault issue reoccurs.

Thanks for your help to date.

If there's anything else you'd like me to try, please do say.

Kind Regards

Chris

Hi,

I really doubt that, if you restarted trisul after setting the BPF, it would never go back and read old IPv6 data.

If the current proceess dies : can you start it in gdb like so.

# gdb /usr/local/bin/trisul
(gdb) r -nodemon /usr/local/etc/trisul/trisulConfig.xml -mode onlinerxring

I am very anxious to track this down.

Thanks,

Hi Vivek,

There was a segfault error overnight, similar to the previous error (dmesg output below).

I've started Trisul up just now using the GDB commands provided thanks. I've not re-enabled either of the config options we disabled previously either.

Kind Regards,

Chris

Just had this message come through from GDB:

[New Thread 0x704ffb70 (LWP 29694)]
[New Thread 0x6fcffb70 (LWP 29695)]
[Thread 0x704ffb70 (LWP 29694) exited]
[Thread 0x6fcffb70 (LWP 29695) exited]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x71affb70 (LWP 29580)]
0x081a0fc0 in ?? ()

Kind Regards

Chris