I've updated the scanner to keep track packets that have been truncated. Until now, headers assumed certain header properties based on the current length of the buffer, which might have been truncated during capture by pcap (i.e. by using snaplen parameter during capture). Although this approach worked OK most of the time, it did not work all the time when dealing with a truncated packet.

I have enhanced the scanner to keep track of both buffer length and original packet wire length, where wire length (wirelen) is the length of the packet during transmission. The scanner keps track of 3 packet length properties: buffer-length, memory-length, wire-length.

Buffer-length is an adjustable buffer lenth, that starts out with the same length as memory-length. For each header that has a postfix, the buffer-length is decremented by that amount, moving the boundary of where the next protocols buffer ends.

Memory-length keeps track of the original length of the buffer in memory. It is not adjustable. This would be the truncated length of the packet.

Wire-length keeps track of theoretical length of the packet. If a packet has been truncated, memory-length - buffer-length equals the number of bytes that were truncated. This property is also decreased with each header prefix processed.

The algortihm is a bit complex to explain here exactly how it works, and I will write up a section on it in the user guide.

In addition to these length properties, the packet and each header also contain a bit-flag property. As the scanner processes packets and headers if any portion of the packet or header needed to be truncated from its protocol specified length, such as when payload is truncated or prefix wiped out by the shorted packet, a flag is in the packet and the specific header.

The packet flag simply states if the packet has been truncated at all (a simple condition that can also be checked by comparing capture-length against wire-length). Each header sets a flag for any header record component that ended up truncated as a result of the shorted packet. There is a separate flag bit for each prefix, header, gap, payload and postfix. The flag bits specify exactly which portions of the packet have been truncated. Convenience methods isPayloadTruncated() etc... are provided that check the appropriate flag bits.

header
[0]: [id=1  ETHERNET   flags=0x4 pre=0 hdr_offset=0    hdr_length=14  gap=0 pay=50  post=0]
[1]: [id=2  IP4        flags=0x4 pre=0 hdr_offset=14   hdr_length=20  gap=0 pay=30  post=0]
[2]: [id=5  UDP        flags=0x4 pre=0 hdr_offset=34   hdr_length=8   gap=0 pay=22  post=0]
[3]: [id=17 RTP        flags=0x4 pre=0 hdr_offset=42   hdr_length=12  gap=0 pay=10  post=0]
[4]: [id=0  PAYLOAD    flags=0x0 pre=0 hdr_offset=54   hdr_length=10  gap=0 pay=0   post=0]

Here is a debug output of a Rtp packet. Its wire-length = 200, but memory-length = 64. It has been truncated. You can see the payload lengths appropriately reduced and the flags set (0x4 = HEADER_PAYLOAD_TRUNCATED flag). The postfix is not marked with a flag as being truncated because in was never defined by any of the protocols. Rtp may have postfix padding, but its optional and this particular packet did not set it, other wise we would also see the HEADER_POSTIFX_TRUNCATED flag set.

In anycase, protocol developers now have to tools neccessary to deal with truncated packets correctly and in every situation, no matter how short the packet has been shorted. Of course this has implications at the higher level's of the API. Analyzers typically might have trouble working with truncated packets, and reassembly might be completely out of the question. This is something that the user will have to take into account when setting up the capture, what they intend to do with the captured packets afterwards. Although for plain analyzers who don't reassemble, but all they need are complete headers, even truncated packets are workable. It will be easy for an analyzer to check if it can proceed with reassembly or analysis. For example a TcpAssembler which reassembles the packets, might use Tcp.isPayloadTruncated check to see if it can proceed with tcp stream reassembly.