How is it possible to regroup packets (for example the same http request is divide in many packets)?
I'm looking for code samples / doc.
How is it possible to regroup packets (for example the same http request is divide in many packets)?
I'm looking for code samples / doc.
I've come up with a better way to handle headers that have a lot of different fields in it. Instead of hard coding methods for every field you can now annotate any Enum table and all the constants within it automatically become fields.
This greatly simplifies long headers such as Http which has hundreds of different possible fields. Here is the entire Http header definition with the new annotation:
public class Http2
extends AbstractMessageHeader {
/**
* HTTP Request fields
*
* @author Mark Bednarczyk
* @author Sly Technologies, Inc.
*/
@Field
public enum Request {
Accept,
Accept_Charset,
Accept_Encoding,
Accept_Ranges,
Authorization,
Cache_Control,
Connection,
Cookie,
Date,
Host,
If_Modified_Since,
If_None_Match,
Referrer,
User_Agent,
RequestVersion,
RequestMethod,
RequestUrl,
}
/**
* HTTP Response fields
*
* @author Mark Bednarczyk
* @author Sly Technologies, Inc.
*/
@Field
public enum Response {
Accept_Ranges,
Age,
Allow,
Cache_Control,
Content_Encoding,
Content_Length,
Content_Location,
Content_Disposition,
Content_MD5,
Content_Range,
Content_Type,
RequestVersion,
ResponseCode,
ResponseCodeMessage,
RequestUrl,
}
@Dynamic(Field.Property.CHECK)
public boolean hasField(Request field) {
return super.hasField(field);
}
@Dynamic(Field.Property.VALUE)
public String fieldValue(Request field) {
return super.fieldValue(String.class, field);
}
@Dynamic(Field.Property.CHECK)
public boolean hasField(Response field) {
return super.hasField(field);
}
@Dynamic(Field.Property.VALUE)
public String fieldValue(Response field) {
return super.fieldValue(String.class, field);
}
@Override
protected void decodeFirstLine(String line) {
String[] c = line.split(" ");
if (c[0].startsWith("HTTP")) {
super.setMessageType(MessageType.RESPONSE);
Its a method that allows you to see who the real owner of the memory your object referring to is. Packet state, header and buffer's get peered to each other it is very possible to loose track of who the owner is.
This debug method JMemory.toDebugString() prints out the following information:
JMemory@b3f004a: size=1506, owner=Pcap.class JMemory@b3f004a: size=1506, owner=Pcap.class JMemory@b3f004a: size=886, owner=Pcap.class JMemory@af7afb0: size=1506, owner=nio.JMemoryPool$Block.class JMemory@b3f004a: size=86, owner=Pcap.class JMemory@b3f004a: size=886, owner=Pcap.class JMemory@b3f004a: size=86, owner=Pcap.class JMemory@af7e788: size=1506, owner=nio.JMemoryPool$Block.class JMemory@af7ed7a: size=886, owner=nio.JMemoryPool$Block.class JMemory@af7f100: size=86, owner=nio.JMemoryPool$Block.class JMemory@af7f166: size=1506, owner=nio.JMemoryPool$Block.class JMemory@af7f758: size=1506, owner=nio.JMemoryPool$Block.class JMemory@af7ffc0: size=886, owner=nio.JMemoryPool$Block.class JMemory@af80346: size=86, owner=nio.JMemoryPool$Block.class [truncated....]
This output is from iterating over a capture file and all its packets. The JMemory.toDebugString() was called in 2 places. First on the original JBuffer that supplied with JBufferHander.nextPacket method and the second time after a new PcapPacket was created with the libpcap buffer copied into it.
You can see 2 owners. Pcap.class and JMemoryPool$Block.class. Pcap class represents memory owned and allocated by libpcap library itself. While the Block.class is the owner using jNetPcap's default allocation scheme while allocates memory in chunks and hands it out for packet copies.
Added 2 address resolvers to JRegistry. IpAddressResolver and IEEEOuiResolver. Ip resolver when enabled resolves and caches IP addresses to hostnames. OUI resolver downloads and caches OUI manufacturer prefix database. JFormatter has been modified to allow IP addresses and MAC addresses to use resolvers. It has new methods for enabling and disabling address resolutions. Default is disabled although it can be changed globally or on a per formatter basis with
static JFormatter.setDefaultResolveAddress(boolean) and instance JFormatter.setResolveAddress(boolean). The first is the global default while the second is per instance.
JRegistry provides setter and getter methods for accessing the address resolvers. Resolvers implement the interface JRegistry.Resolver. Therefore custom algorithms and resolver types can be added replacing default ones.
Here is what output from Ethernet and Ip4 headers looks like now:
Eth: ******* Ethernet - "Ethernet" - offset=0 (0x0) length=14 Eth: Eth: destination = SMC_22:5a:03 (00:04:e2:22:5a:03) Eth: .... ..0. .... .... = [0] LG bit Eth: .... ...0 .... .... = [0] IG bit Eth: source = KYE_20:6c:df (00:c0:df:20:6c:df) Eth: .... ..0. .... .... = [0] LG bit Eth: .... ...0 .... .... = [0] IG bit Eth: type = 0x800 (2048) [ip version 4] Eth: Ip: ******* Ip4 - "ip version 4" - offset=14 (0xE) length=20 Ip: Ip: version = 4 Ip: hlen = 5 [5 * 4 = 20 bytes, No Ip Options] Ip: diffserv = 0x0 (0) Ip: 0000 00.. = [0] code point: not set Ip: .... ..0. = [0] ECN bit: not set Ip: .... ...0 = [0] ECE bit: not set Ip: length = 475 Ip: id = 0xC139 (49465) Ip: flags = 0x2 (2) Ip: 0.. = [0] reserved Ip: .1. = [1] do not fragment: set
I've done the Http header and its one of those Mime based headers that can have many dozen's of different types of fields. It seems unreasonable to have the header author define everyone of those fields as a separate accessor method. We need a better header container for those type of headers (http, SIP, Mail message, etc..)
The existing Http definition (found under tests directory), already keeps track of all those headers in a Map<String, String>. There is an accessor method to retrieve the map and then using normal map methods you can check for each field. I think that is the right approach since there are so many possible fields, and most of them are almost always not there. The JHeader.decodeHeader method properly sets the offset and length properties for each Http field, thus it is simply a matter of a normal numerical sort to get the present fields to display properly in a formatter.
A single AbstractMimeHeader baseclass can take care of all of those protocols at once, including http.
One currently unresolved issue with Http is the multi-part contents. I've been thinking about this quiet a bit, and it doesn't make any sense to implement multiparts as hardcoded embeded sub-headers, although that is how they appear. What I'm leaning towards is a new type of sub-header that can act as as a proxy (a surrogate) and place actual standalone headers as sub-headers. This will allow for example Http header to be completely standalone as a header after http and as an embeded sub-header for Http header's with multiparts.
New text handling in JBuffer seems to be working well for Http so that part is done and working.
Just some of the things I'm working on and thinking about.
I found 3 new bugs in rc3. They have been fixed.
hello everyone.
i have a problem in sending the packet that i have overwritten.
the code is below
pcap.dispatch(1000, new PcapHandler() { public void nextPacket(String user, long seconds, int useconds, int caplen, int len, ByteBuffer buffer) { buffer.put(getByteMAC(MAC_NAS));//newdes buffer.put(getByteMAC(MAC_LAPTOP_WIRE));//new source for(int jj=0 ;jj<5;jj++){ System.out.println("jj is "+jj); pcap.sendPacket(buffer.array()); } } }
the console output is like this
jj is 0 (there is no loop in there)
but when i didnt send the packet, the loop worked well
jj is 0, jj is 1 ....
note: i have checked that i could send using pcap.sendPacket(buffer.array()), with arp packet that i've created and overwritten it.
anyone know why i couldnt send the packet?
Best Regards
Timmy
Here is some information about jNetPcap internals, how it works and how classes related to each other. This is not intended for typical user use, but for developers that need to know how the library functions internally.
There are 2 parts to jNP, the libpcap wrapper and the packet decoding framework. The libpcap wrapper doesn't have much logic, it truely is just a wrapper that takes java calls and via the "native" modifier on java method, binds JNI (Java Native Interface) code to java code.
Static methods, have not persistent state and simply query some kind of global pcap function. Instance methods such as the ones in Pcap and WinPcap classes that first require pcap handle to be acquired, store the pcap structure's physical address in a private field in Pcap class. This field is retrieved everytime an instance method is invoked and converted to a memory pointer. Type cast to appropriate structure and used in the original native call. The result is returned and that all there is too it. All libpcap wrapper function work that way. There is very little other logic at JNI level for libpcap wrapper functions. Exceptions are findAllDevs method which converts a linked list of pcap_if structures to a java list. The same goes for addresses retrieved from those structures.
Then we get to packet decoding framework. This is everything under org.jnetpcap.packet package.
Hi *,
I wrote a custom header (ieee 802.11 radiotap) which worked fine until rc3.
i did change all the annotations. its compiling, even decoding all those optional subheaders (writing correct values in optionsBitmap, optionsLengths and optionsOffsets).
but with rc3 i'm getting a nullpointer-exception
Opening: /home/[... snip ...]/captures/radiotap_IEEE80211_LLC_IP4_TCP Protocol for this DLT [(127) which is (IEEE802_11_RADIO)] --> [org.jnetpcap.packet.header.IEEE802dot11_RADIOTAP] Reading 1 packet... Packet caplen=104 wirelen=104 id=1 name=TSFT is_present=0 present_bitmap=0x482E optionsBitmap=0x0 pad=0 offset of next field=8 length=0 [... snip ...] id=14 name=DB_ANTENNA_NOISE is_present=0 present_bitmap=0x482E optionsBitmap=0x105C pad=0 offset of next field=16 length=0 Exception in thread "main" java.lang.NullPointerException at org.jnetpcap.packet.structure.JField.getLength(JField.java:238) at org.jnetpcap.packet.format.JFormatter.stylizeBitField(JFormatter.java:508) at org.jnetpcap.packet.format.JFormatter.stylizeSingleLine(JFormatter.java:609) at org.jnetpcap.packet.format.TextFormatter.fieldBefore(TextFormatter.java:94) at org.jnetpcap.packet.format.JFormatter.format(JFormatter.java:274) at org.jnetpcap.packet.format.JFormatter.format(JFormatter.java:278) at org.jnetpcap.packet.format.JFormatter.format(JFormatter.java:304) at org.jnetpcap.packet.format.JFormatter.format(JFormatter.java:254) at org.jnetpcap.packet.format.JFormatter.format(JFormatter.java:347) at org.jnetpcap.packet.format.JFormatter.format(JFormatter.java:316) at org.jnetpcap.packet.JPacket.toString(JPacket.java:694) at de.fuberlin.mi.cst.WiFiClipboard.WiFiClipboard.nextPacket(WiFiClipboard.java:39) at org.jnetpcap.Pcap.loop(Native Method) at org.jnetpcap.Pcap.loop(Pcap.java:1947) at de.fuberlin.mi.cst.WiFiClipboard.WiFiClipboard.testRadiotap(WiFiClipboard.java:19)
Released rc3 today. There were still minor things I didn't get a chance to do before the release, but after fixing that major memory leak, I couldn't delay the release any longer. Better release more frequently with updates.
The rc3 adds major support for custom headers. Annotations, JRegistry and behind the scenes JHeaderScanner and JScanner are working very well so far.
I think once I get all the feedback on rc3 and if there are no more major bugs or API issues, I plan on finanalizing rc3 and releasing it as the official production 1.2 release. If there are any bugs or significant API updates we'll have 1.2.rc4, but I'm hoping to avoid it now that code is starting to stabilize and as far as I can tell the API is functioning very well both from performance and ease of use perspective.
I renamed the annotation @FieldRuntime to @Dynamic and also moved FieldRuntime.FieldFunction and renamed it to Field.Property. Property is an enum table and now dynamic properties are annotated in a way that is actually meaningful in itself. Here is what I mean:
@Dynamic(Field.Property.DESCRIPTION)
public String fieldADescription() {
return "My Description";
}
Before it would have said:
@FieldRuntime(FieldFunction.DESCRIPTION)
The new way is much more verbose and right to the point.
I tackled the Http and Html protocol headers as they are text based and stress different portions of the protocol development API. Glad I did as that highlighted some deficiencies in JBuffer. JBuffer didn't have any meaningful support for strings that may be contained in the buffer. Added a number of string related methods that now allows scanning for string patterns and string extractions from the buffer.