jNetPcap v2 - Architecture

No replies
Mark Bednarczyk
Offline
Joined: 03/22/2008

I'd like to start with the following: The API needs to be redesigned, modular and cleaned up.

I think that says it all for version 2 of jnetpcap. Below is the architectural overview we propose. A lot of this comes from things we wanted to do, need to do, experience with our commercial software and the direction where technology is going.

Modular

The jNetPcap project needs to be implemented as a set of coherent modules that add functionality on top of a 'core' feature set. This will allow modules to be updated inpendently without affecting other components of the overall software.

It will also allow user's of the library to pick and choose the modules which they need and not have to download every thing every time.

In terms of development focus, this is also very beneficial, as teams can break off and work on individual modules. We should not leave out, that a modular system will allow specific groups or even companies to take development and maintenance ownership at a modular level.

Lastly, lest not forget that commercial opportunities will present themselves for providing alternatives to the default modules provided by the project, or even modules that are only available commercially, if there is no one else developing or contributing a particular functionality.

A modular system will allow great many different varieties and choices for user's of jNetPcap library.

Proposed Modules

Its an early stage of planning, but the following might provide guidance and example of what type of modules we could develop.

  1. PCAP wrapper - base module that wraps as a thin layer the unix/windows libpcap APIs.
  2. PF_RING wrapper - a PF_RING module which enhances the performance of libpcap
  3. Intel's DPDK - Data Plane Development Kit
  4. OpenSOC - play nice with OpenSOC standard
  5. Wireshark plugin - allows jnetpcap modules to act as a plugin for wireshark allowing synergy between the power of wireshark and jnetpcap protocol development.
  6. Offline file processing using jNetStream - a not well known predecessor to the current jNetPcap project, which has amazing, file handling capabilities. Allows inplace edits and sophisticated rearrangement of capture file contents. As for the rest of jnetstream capabilities, that can be scrapped since it will be provided by jnetpcap.
  7. Core protocol group - establish a set of core protocols supported, DPI, SPA (Stateful Packet Analysis), events, etc...
  8. Various additional groups of protocols

All of these modules would be optional with depending on user's need.

Features

In terms of features, this is something that needs further discussion, but a few come to mind based on existing jNetPcap functionality and the most popular requests for new features.

  1. Full libpcap API - broken out by various libpcap API version numbers.
  2. Extensible callback mechanism - a flexible callback mechanism based on "Callback" interface. This is in the middle interface between libpcap dispatch mechanism and java listeners/handlers implemented by users. The "Callback" implementation provides actuall dispatching to listeners and implements its own native implementation for transitioning between libpcap dispatch to the java dispatch. For example, one "Callback" implementation can provide a series of ZERO copy dispatcher methods to a particular listener interface, while another implementation can provide "buffered" dispatcher methods to the same listener interface while providing additional ones that take advantage of the buffered packet data. This provides extensive amount of flexibility and extensibility in the future.
  3. Efficient memory management - utilize pre-allocated buffers for everything jnetpcap does. This allows the user to control how much memory is used based on the tasks at hand. When buffers overfull then artifacts (packets, events, streams, etc) are dropped and appropriate drop counters are incremented as feedback to let the user know that artifacts are being dropped which will require tuning of the application. This is a much more sensible and industry accepted method of high performance applications.
  4. DPI - deep packet inspection. Full DPI capability on a per packet basis.
  5. SPA - state-full packet analysis and reassembly. Allows related packets such as in TCP streams to be analyzed with state-full structures, leading to reassembly of.
  6. L4-L7 protocols - these protocols are only after underlying transport layer data has been reassembled. This is a whole category of new features in itself.
  7. Module management - framework for modularization of the API.

Libpcap APIs are constantly evolving and are nicely versioned. The 1.X version of jNetPcap already breaks the APIs by version number. That is for future and backward libpcap compatibility, allow jnetpcap gracefully downgrade API to match APIs and features of the runtime libpcap library installed. This is currently done for 2 levels of libpcap API version 0.8.0 - 1.0.0. However it needs to be continued for each new libpcap release and update. A straight forward static call to check on API availability such as Pcap.isAPILevelAvailable(String version): boolean could be used to easily determine with a particular call is actually implemented by the currently installed native libpcap library.

Sly Technologies, Inc.
http://slytechs.com