Release 1.2.rc5 is released to SourceForge.net release system.
The latest release is a maintenance and new feature release. The release fixes several bugs found since 1.2.rc4. It also adds significant new capability for high level protocol analysis. Analyzers now keep track of sessions, segments and fragments. Provides services to reassemble tcp segnements and Ip fragments. Release also adds Http and Html protocol definitions and supporting analyzers.
* Fixes
- #2688877 - reported that debian package is missing .so soft link to .so.rc4
main lib file. Will add that link in post-install to debian
package control file
- #2648427 - PcapDumper.dump() core-dumps. A patch is available upon request.
- #2557009 - LLC header problem
* Added
- Pcap.loop() and Pcap.dispatch() variation to allow the user to supply
JScanner. This allows JScanner internal buffer size to be changed by the
user
- Added JPacket.setMemoryPool(). This allows the user to change the default
allocation block size of JMemoryPool for packets that are copied via the
constructor and transferTo() methods
- Added analyzer support for Tcp/Ip family of protocols. This includes Ip
fragment reassembly, tcp segment sequencing and reassembly as well
- Added Http and Html as CORE jnetpcap protocols supported by native scanner
* Changed
- Moved org.jnetpcap.packet.header package to new org.jnetpcap.protocol
package. Protocols are grouped into families of protocols, which are stored
under appropriate protocol suite sub-packages.
The change log notes forgot to also include:
* Flow tagging - each header is tagged with flow information. You can extract the flow key using JPacket.getState().getFlowKey: JFlowKey. Flowkeys are unique network traffic flows, similar to Cisco NetFlow. Flows can be grouped and mapped. Flows are different from Tcp and Ip4 sequencing and analysis.
Comments
TcpStream
Hi Mark,
I have a PCAP file, where multiple HTTP messages are segmented over a number of TCP packets, within the same TCP connection.
In jNetPcap rc4, I was having to manually buffer the contents of each segmented HTTP message until it was complete. It was a clunky way of solving my problem.
Therefore in jNetPCap rc5, I am hoping to execute... Http http = packet.getHeader(new Http()); Where the http object returned is the complete HTTP message, even if its segmented.
Does it work like this, as currently Im using the nextPacket loop, which steps through the PCAP file frame by frame.
Or do I have to use this new TcpStream class ?
If you could provide example code that would be great.
Regards.
Yes that is how it almost
Yes that is how it almost works. TcpStream and TcpDuplexStream contain information about the stream but do not do any reassembly themselves. The contains persistent state for the entire duration of the stream. Useful stuff like begining sequence numbers, MSS info and others in each direction. I'm working on few examples to demonstrate how to use this.
I say almost because the fragments are not modified, but they have analysis information attached to the original packet. The analysis info for reassembly is new packet with contains the original Http request/reply and all of the reassembled segments that are part of it.
The Http protocol requests from tcp layer reassembly. Tcp analyzer buffers packets, monitors TCP acks and the reassembled acknowledged tcp segments. Then passes the reassembled buffer upto Http which builds a new Http packet and attaches it to the original Http packet that triggered the reassembly.
Sly Technologies, Inc.
R&D
I should add there is also an
I should add there is also an alternative. You can register a handler with HttpAnalyzer and it will dispatch to you complete Http requests and responses. So if you are not interested in lower level headers then working directly with Http protocol layer is the better approach.
All the analyzer's provides handlers:
and so forth.
Sly Technologies, Inc.
R&D
Thanks. Yes it seems like the
Thanks. Yes it seems like the HttpAnalyzer would be a better design.
So for example if you provided a SIP "protocol class", would this mean you would also provide a SipAnalyzer class?
Sip will require a
Sip will require a SipAnalyzer to tell Tcp how to reassemble its requests/responses. SipAnalyzer and HttpAnalyzer should be very similar and probably can use common abstract analyzer class.
When you think about it from protocol perspective, Tcp layer has no idea which segments are what and how they should be assembled, except as a stream of bytes. Higher level protocols such as Sip and Http have "Content-Length" fields that tell them how much data to reassemble. Further more they know if they are a request, response or something else.
I'm pretty sure I have the analyzer architecture right. I'd appreciate any feedback on that. I'm still finding bugs and issues in the implementation of it though. Some new test cases I started writing yesterday are failing, I'm digging into it now.
HttpAnalyzer, TcpAnalyzer, Ip4Analyzer work OK under certain circumstances. I still have work to do to make it robust under all circumstances. Now that I have the architecture complete which took few weeks to design and first cut at the implementation working, its now a matter of making it robust. Work should go a lot faster then between rc4 to rc5.
I need to add a chapter to the user guide to go over what that architecture is. It should make more sense after that how various components of a "protocol" (that is, headers, analyzers, events, listeners, utility classes i.e. HttpParser and exceptions) fit together. Because of all these components that make up a protocol, I had to move everything into a more "protocol" centric java package scheme. The plain org.jnetpcap.packet.header package wasn't enough and things were getting crowded very fast.
Sip and Http protocols are very similar. I created a common "AbstractMessageHeader" class that does most of the work for Http and for Sip headers. I will do the same with HttpAnalyzer and SipAnalyzers, such as "AbstractMessageAnalzyer" for example. Once Http is fully working, it should be no problem to just add Sip and any other mime/hyper-text based protocol.
Anything higher level then Sip or Http is a breeze, like JPEG images, or any type of content. HttpAnalyzer has already taken care of the details of sequencing and assembly. I have a test case with JPEG image header carried over http, its just like any other header, except when you consider all the behind the scene work that Ip4Analyzer, Ip4Sequencer, Ip4Assembler, TcpAssember, TcpSequencer, TcpAssember, HttpAnalyzer had to do to reassembled ip fragments into tcp segments, tcp segments into Http request/response commands.
We're getting there.
Sly Technologies, Inc.
R&D
Some new apps that were
Some new apps that were using with our call center software are running on this.