Breakout of the infinite pcap.loop after certain number of minutes

13 replies [Last post]
ambudsharma
Offline
Joined: 08/09/2009

Hi Mark,
I am trying to write a program that dumps packets to a pcap file and after certain number of minutes, saves the file. Another file is then opened and the oncoming packets are dumped into this second file for the same number of minutes.

This is done over and over. The resulting file-set is a collection of .pcap files each of exact same number of minutes. They can later be analysed.

The main code looks like this:

final PcapDumper dumper = pcap.dumpOpen(ofile);
final PcapHandler dumpHandler = new PcapHandler() {
	public void nextPacket(PcapDumper dumper, long seconds, int useconds, int caplen, int len, ByteBuffer buffer) {
			   dumper.dump(seconds, useconds, caplen, len, buffer);
			}
		};
pcap.loop(pcap.LOOP_INFINATE,dumpHandler, dumper);

But when I use pcap.LOOP_INFINATE, the program goes into infinite loop and I dont know the correct approach to create the file-sets as I mentioned above. Is there a way to capture packets to a file only for certain number of minutes and perform this task over and over?

Thanks,
-Bud

ohaya
Offline
Joined: 07/13/2009
Hi, I only have minimal

Hi,

I only have minimal experience with using pcap.loop(), but when you use that, doesn't the nextPacket() get called for each packet? If so, can you not then call dumper.close() inside nextPacket() and then have something like in nextPacket():


.
.
dumper.close();
ofile = ;
PcapDumper dumper = pcap.dumpOpen(ofile);
.
.

??

Like I said, not sure, but just a thought...

Jim

Mark Bednarczyk
Offline
Joined: 03/22/2008
Probably not a good idea to

Probably not a good idea to call Pcap.close from inside the handler. You should first break out of the loop using Pcap.breakLoop call, then when you've broken out, do the close on the outside.

You also want to specify a timeout on the Pcap.openLive, to make sure that the loop does not block forever waiting for a packet to arrive which would not give you an opportunity to issue Pcap.breakLoop.

so most likely you have to put an outside loop around your Pcap.loop with a timeout. You can reenter the loop as many times as needed. Use an AutomicBoolean object as a flag in the loop to break out and also check in the outside loop for break-out condition and not reenter the outside loop:


final AtomicBoolean flag = new AutomicBoolean(true);
final Pcap pcap = ...; // openLive with a timeout

while(flags.get() && Pcap.loop(Pcap.INFINITE_LOOP, handler, dumper) == Pcap.OK) {
;;
}

pcap.close();

And in handler just check for breakout flag:

if (flag.get() == false) {
pcap.breakLoop();
}

Hint: don't forget the final keywords so you can reference those objects within the handler.

Note: that with build 1.3.b0005 API now as Pcap.loop(int cnt, PcapDumper) special case call, that bypasses the normal java handler, and does the dumping for you all in native space. You would not be able to insert the breakLoop code into the handler, but if you keep the timeouts fairely short, you may just get away with doing the check in the outside loop. This would give you better performance.

Sly Technologies, Inc.
Main website: http://slytechs.com
Customer portal: http://support.slytechs.com

Come see Sly Technologies at
Ethernet Technology Summit in Santa Clara, Ca
Booth #309 April 29 - May 1, 2014

ambudsharma
Offline
Joined: 08/09/2009
thanks

thanks Jim. I'll try this

ambudsharma
Offline
Joined: 08/09/2009
the flow of control

Mark,
The follow of control for what I am trying to do is as follows. I am new to packet programming in JAVA and thus need a little more insight on how to go about this. I understood your last post, but I still am pretty confused as to how to go about putting the different pieces of this puzzle together.

Bird's eye view of Flow of control:
Step 1: File created
Step 2: Dumper activated
Step 3: ALL Packets dumped into step1 file for 5 minutes
Step 4: Break out of loop in step 3.
Step 5: close file in Step 1.
Step 6: Go to step 1 and repeat step 1 to step 5 infinitely.

Code:

public class pcapDumperTest {
	public static final String DATE_FORMAT_NOW = "yyyyMMddHHmmss";

	@SuppressWarnings("deprecation")
	public static void main(String[] args) {
		List alldevs = new ArrayList(); // Will be filled with NICs
		final StringBuilder errbuf = new StringBuilder(); // For any error msgs
		int r = Pcap.findAllDevs(alldevs, errbuf);
		if (r == Pcap.NOT_OK || alldevs.isEmpty()) {
			System.err.printf("Can't read list of devices, error is %s", errbuf.toString());
			return;
		}
		int i = 0;  
		for (PcapIf device : alldevs) {  
			System.out.printf("#%d: %s [%s]\n", i++, device.getName(), device.getDescription());  
		}  
		System.out.println(alldevs.get(1));
		final PcapIf device = alldevs.get(1); // We know we have atleast 1 device
		final int snaplen = 64 * 1024; // Capture all packets, no trucation
		final int flags = Pcap.MODE_PROMISCUOUS; // capture all packets
		final int timeout = 10 * 1000; // 10 seconds in millis
		final AtomicBoolean flag = new AtomicBoolean(true);
		final Pcap pcap = Pcap.openLive(device.getName(), snaplen, flags, timeout, errbuf);
		if (pcap == null) {
			System.err.printf("Error while opening device for capture: " + errbuf.toString());
			return;
		}

				//System.out.println(now());
				String ofile = "pcap" + now().toString()+".pcap";
				File file = new File(ofile);
				final PcapDumper dumper = pcap.dumpOpen(ofile); // out file

				final PcapHandler dumpHandler = new PcapHandler() {
					public void nextPacket(PcapDumper dumper, long seconds, int useconds, int caplen, int len, ByteBuffer buffer) {
						dumper.dump(seconds, useconds, caplen, len, buffer);
						if (flag.get() == false) {
							pcap.breakloop();
						}
					}
				};



				pcap.loop(pcap.LOOP_INFINATE,dumpHandler, dumper);


				while(flag.get() &&  pcap.loop(Pcap.LOOP_INFINATE, dumpHandler, dumper) == Pcap.OK) {
					;;
				}

				pcap.close();
				System.out.println(ofile);
				dumper.close(); 
				//pcap.close();

	}
	
	public static String now() {
		Calendar cal = Calendar.getInstance();
		SimpleDateFormat df = new SimpleDateFormat(DATE_FORMAT_NOW);
		return df.format(cal.getTime());
	}

}

I have removed the timer function from the code above and I can add it later. I am pretty confused as of now since maybe I dont understand the flow of control for packet captures to a file.

Your help is greatly appreciated!
Thanks,
-Bud

ohaya
Offline
Joined: 07/13/2009
Hi ambudsharma, As one

Hi ambudsharma,

As one "beginner" to another, I can appreciate what you're not understanding Smile...

I think that jNetPcap kind of reflects the libpcap API, i.e., it's mostly a "Java-friendly" wrapper for libpcap, via JNI. That's the impression I get, anyway.

So, given that, some of the programming concepts are either the same or similar to "native" libpcap programming, albeit in Java, rather than in C or say, Python.

For the libpcap programming (in C), there's a pretty good tutorial at:

http://www.tcpdump.org/pcap.htm

The part you probably want to focus on is the section labelled "The actual sniffing", where he explains about how .loop()/.next() work using callbacks.

BTW, in my case, I'm not working with live capture (from a network) so I don't have the kind of timing dependencies that would be run into with live capture, so I've chosen to not use the .loop()/.next(), but rather use .nextEx(), which is much easier (for me, at least) to understand.

However, since you're looking to do live capture, this may or may not be an option, because of timing, etc.

Hope that helps...

Jim

Mark Bednarczyk
Offline
Joined: 03/22/2008
You are on the right track.

You are on the right track. Your program looks correct. It should work. Please explain the kind of problem you are experiencing if its not working for you.

You also should review the difference between Pcap.loop and Pcap.dispatch methods. They differ how they interpret packet cnt and timeout parameters. If you are still having trouble, I will write up a demo app that will do exactly what you are trying to do, as this kind of example will be beneficial to others as well.

Sly Technologies, Inc.
Main website: http://slytechs.com
Customer portal: http://support.slytechs.com

Come see Sly Technologies at
Ethernet Technology Summit in Santa Clara, Ca
Booth #309 April 29 - May 1, 2014

ambudsharma
Offline
Joined: 08/09/2009
Follow up - not working

Hi Mark and Jim,
I was able to understand how things are implemented. I tried to do a trace analysis of the flow of control of the program. I have put forth my problems after the code. Here's my updated program:

public class pcapDumperTest {
	public static final String DATE_FORMAT_NOW = "yyyyMMddHHmmss";
	public static AtomicBoolean exitFlag = new AtomicBoolean(true);

	@SuppressWarnings("deprecation")
	public static void main(String[] args) {
		List alldevs = new ArrayList(); // Will be filled with NICs
		final StringBuilder errbuf = new StringBuilder(); // For any error msgs
		Timer timer = new Timer();
		int r = Pcap.findAllDevs(alldevs, errbuf);
		if (r == Pcap.NOT_OK || alldevs.isEmpty()) {
			System.err.printf("Can't read list of devices, error is %s", errbuf.toString());
			return;
		}
		int i = 0;  
		for (PcapIf device : alldevs) {  
			System.out.printf("#%d: %s [%s]\n", i++, device.getName(), device.getDescription());  
		}  
		System.out.println(alldevs.get(1));
		final PcapIf device = alldevs.get(1); // We know we have atleast 1 device
		final int snaplen = 64 * 1024; // Capture all packets, no trucation
		final int flags = Pcap.MODE_PROMISCUOUS; // capture all packets
		final int timeout = 10 * 1000; // 10 seconds in millis
		final AtomicBoolean flag = new AtomicBoolean(true);
		final Pcap pcap = Pcap.openLive(device.getName(), snaplen, flags, timeout, errbuf);
		if (pcap == null) {
			System.err.printf("Error while opening device for capture: " + errbuf.toString());
			return;
		}
		

		final PcapHandler dumpHandler = new PcapHandler() {
			public void nextPacket(PcapDumper dumper, long seconds, int useconds, int caplen, int len, ByteBuffer buffer) {

				dumper.dump(seconds, useconds, caplen, len, buffer);
			}
			
		};
		while(exitFlag.get()) {
			String ofile = "pcap" + now().toString()+".pcap";
			File file = new File(ofile);
			PcapDumper dumper = pcap.dumpOpen(ofile);
		pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);
		}
    	pcap.close(); 
		System.exit(0);
		
	}

	public static String now() {
		Calendar cal = Calendar.getInstance();
		SimpleDateFormat df = new SimpleDateFormat(DATE_FORMAT_NOW);
		return df.format(cal.getTime());
	}

}

The program puns fine. This is what I understand. When the program starts, it goes into the following segment of the code:

while(exitFlag.get()) {
			String ofile = "pcap" + now().toString()+".pcap";
			File file = new File(ofile);
			PcapDumper dumper = pcap.dumpOpen(ofile);
		pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);
		}


Then, after 10 seconds being in "pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);" it hands over the control to:


	final PcapHandler dumpHandler = new PcapHandler() {
			public void nextPacket(PcapDumper dumper, long seconds, int useconds, int caplen, int len, ByteBuffer buffer) {

				dumper.dump(seconds, useconds, caplen, len, buffer);
			}
			
		};

The above code writes the packets captured to the file created.

This is what I am trying to accomplish: every time it times out (in this case 10 seconds), I need the handler to just write the packets to the file and close the file. It should hand over control to "pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);", wait for 10 seconds, timeout, create another file (different one) and handler should write all the packets captured in last step to the file. I can then increase the timeout to 5 minutes and be all done. If I run this over and over, I should have 5 minute captures in different files with their respective timestamps in their file names.

The problem is:
Everytime the timeout occurs, the handler comes in and takes over the control and writes the packets. In the meantime, the packets are being captured which will be written to a file in the next "handler" iteration. The problem is, when the handler hands over the control after writing the packets through "dumper.dump(seconds, useconds, caplen, len, buffer);", it STRAIGHT AWAY goes to "pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);" bypassing everything in the middle! Then the program is just a hand-over between capture and handler (for writing to the file) with a timeout of 10 seconds - over and over and over! I need it to bail out of the handler loop after writing packets captured in the last 5 minutes, close the file and create a new one for writing packets after each timeout from the "capture code" i.e. "pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);".

Algorithm:
1.)Start program, set up device etc
2.) Start capture with --- pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);
3.) When the timeout occurs, transfer control to handler
4.) handler creates a new file and writes all packets to it
5.) handler closes the file and saves it
6.) handler transfers control to "capture" loop --- pcap.loop(Pcap.LOOP_INFINATE,dumpHandler, dumper);
7.) go to step 2

Please help! I dont understand how it is bypassing all code and just "directly" passing over control between start_capture --> hit timeout --> handler write the captured packets --> goes back to start_capture --> hit timeout --> so on and so forth..

Thanks in advance guys!

ohaya
Offline
Joined: 07/13/2009
Hi, Based on his having WAY

Hi,

Based on his having WAY more experience with this stuff, Mark would know the answer to this (my) question, but I'm wondering: Given that ambudsharma is doing live capture, how much of a problem, performance-wise, would it be if he switched to using nextEx() instead of pcap.loop() or pcap.dispatch()? Would he start missing/losing packets? I keep thinking that it'd be (logically) easier to do what he wants to do using nextEx() Smile...

Jim

Mark Bednarczyk
Offline
Joined: 03/22/2008
Here is an example that I

Here is an example that I wrote up that does what we wanted:
http://jnetpcap.svn.sourceforge.net/viewvc/jnetpcap/jnetpcap/trunk/examp...

I mislead you with openLive and timeout. Pcap.loop does not exit on a timeout. The timeout simply tells libpcap to dispatch its packets to user handler even though it would like to buffer some more. Its more intended from interactive traffic that you wouldn't want blocking when packets are arriving simply because libpcap's buffer isn't full.

The only reliable way is to use Pcap.breakloop to break out of Pcap.dispatch and Pcap.loop.

The example work for busy networks, for networks that don't receive any packets the example will not create new files, until at least one packet arrives. A better approach might be not to break out of the loop, but create the dumper within the handler and switch it there.

Unfortunately there is no way to interrupt libpcap when its capturing and there are no packets arriving. Pcap.breakloop only sets a flag in libpcap and the flag is only checked when a packet arrives. So if no packets are arriving, the loop won't break even after break loop was issued.

Sly Technologies, Inc.
Main website: http://slytechs.com
Customer portal: http://support.slytechs.com

Come see Sly Technologies at
Ethernet Technology Summit in Santa Clara, Ca
Booth #309 April 29 - May 1, 2014

ambudsharma
Offline
Joined: 08/09/2009
Thanks!

Hey Mike and Jim,
Thanks for all your support! I am very grateful. I am trying to do something cool here using this API and when I am done, I'll post over the code for everyone to benefit from it. In the meantime, if I have concerns/doubts, I'll ring them here!
Thanks again,
-Bud

sunnyhay
Offline
Joined: 03/05/2010
Hi Mark, I copied your

Hi Mark,

I copied your example and ran it in my Fedora. Sadly I found still infinite loop when capturing packets. The Jnetpcap version is 2.0.
[root@localhost Download]# rpm -qa |grep 'jnetpcap'
jnetpcap-2.0.b0001-1.fc9.i386
I run it in Elimpse with JDK1.6 lib.

Here is the whole code.

/**
 * Copyright (C) 2008 Sly Technologies, Inc. This library is free software; you
 * can redistribute it and/or modify it under the terms of the GNU Lesser
 * General Public License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version. This
 * library is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
 * A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
 * details. You should have received a copy of the GNU Lesser General Public
 * License along with this library; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 */
package Examples;

import java.io.File;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.List;

import org.jnetpcap.JBufferHandler;
import org.jnetpcap.Pcap;
import org.jnetpcap.PcapDumper;
import org.jnetpcap.PcapHeader;
import org.jnetpcap.PcapIf;
import org.jnetpcap.nio.JBuffer;

/**
 * This example is uses pcap library to capture live packets and dump them to a
 * set of files. Packets are captured for a certain amount of time and dumped to
 * a file. After the time interval expires a new capture files is created and
 * next set of packets is dumped for that same time interval and so on.
 * 
 * @author Mark Bednarczyk
 * @author Sly Technologies, Inc.
 */
public class MyTimer {

	public static final String DATE_FORMAT_NOW = "yyyyMMddHHmmss";
	public static final int CAPTURE_INTERVAL = 10 * 1000; // 10 seconds in
															// millis

	public static void main(String[] args) {
		List alldevs = new ArrayList(); // Will be filled with
		// NICs
		StringBuilder errbuf = new StringBuilder(); // For any error msgs

		/***************************************************************************
		 * First get a list of devices on this system
		 **************************************************************************/
		int r = Pcap.findAllDevs(alldevs, errbuf);
		if (r == Pcap.NOT_OK || alldevs.isEmpty()) {
			System.err.printf("Can't read list of devices, error is %s", errbuf
					.toString());
			return;
		}
		PcapIf device = alldevs.get(0); // We know we have at least 1 device

		/***************************************************************************
		 * Second we open up the selected device
		 **************************************************************************/
		int snaplen = 64 * 1024; // Capture all packets, no truncation
		int flags = Pcap.MODE_PROMISCUOUS; // capture all packets
		int timeout = CAPTURE_INTERVAL; // No timeout, non-interactive traffic
		Pcap pcap = Pcap.openLive(device.getName(), snaplen, flags, timeout,
				errbuf);
		if (pcap == null) {
			System.err.printf("Error while opening device for capture: "
					+ errbuf.toString());
			return;
		}

		/***************************************************************************
		 * Thirds we create a main loop which will loop over all capture intervals
		 * and create the necessary dumper for each loop
		 **************************************************************************/

		try {
			while (true) {

				String ofile = "/home/harry/Download/pcap" + now().toString() + ".pcap";
				final PcapDumper dumper = pcap.dumpOpen(ofile);
				final long interval = System.currentTimeMillis() + CAPTURE_INTERVAL;
				
				System.out.printf("new dump files = %s\n", ofile);

				/***************************************************************************
				 * Fourth we create a packet hander which receive packets and
				 * tell the dumper to write those packets to its output file
				 **************************************************************************/
				JBufferHandler dumpHandler = new JBufferHandler() {

					public void nextPacket(PcapHeader header, JBuffer buffer,
							Pcap pcap) {
						
						dumper.dump(header, buffer);
						
						if (System.currentTimeMillis() > interval) {
							pcap.breakloop();
						}

					}
				};

				/***************************************************************************
				 * Fifth we enter the loop and tell it to capture 10 packets. We
				 * pass in the dumper created in step 3
				 **************************************************************************/
				pcap.loop(Pcap.LOOP_INFINATE, dumpHandler, pcap);

				dumper.close(); // close out the dumper and flush any unwritten packets
			}
		} finally {

			/*
			 * Last thing to do is close the dumper and pcap handles
			 */
			pcap.close();
		}

	}

	public static String now() {
		Calendar cal = Calendar.getInstance();
		SimpleDateFormat df = new SimpleDateFormat(DATE_FORMAT_NOW);
		return df.format(cal.getTime());
	}

}
sunnyhay
Offline
Joined: 03/05/2010
I try to debug the code.

I try to debug the code. Set a break point at the line "pcap.breakloop();" and find that every 10 seconds the breakloop does take place. But then the program continues the next line to execute, which is exactly "pcap.loop(Pcap.LOOP_INFINATE, jpacketHandler, "Rock"); ".
So every 10 seconds, the program breaks the loop and then goes on with the loop. That's an infinite loop.
I try also to put "pcap.close();
System.out.printf("Now the pcap exits normally!\n");" after "pcap.breakloop();"

that is

if (System.currentTimeMillis() > interval) {
pcap.breakloop();
pcap.close();
System.out.printf("Now the pcap exits normally!\n");
}

But the loop finally jumps out in 10 seconds but with an disgraceful way:
Exception in thread "main" Now the pcap exits normally!

org.jnetpcap.PcapClosedException
	at org.jnetpcap.Pcap.datalink(Native Method)
	at org.jnetpcap.Pcap.datalinkToId(Pcap.java:1012)
	at org.jnetpcap.Pcap.loop(Pcap.java:2422)
	at Examples.NewExample.start(NewExample.java:145)
	at Examples.NewExample.main(NewExample.java:168)

So how to deal with the problem, Mark?
Thank you! I think the problem is common for any user who wants capturing live packets in a constant period, say 10 seconds or more.

Mark Bednarczyk
Offline
Joined: 03/22/2008
Unfortunately with libpcap,

Unfortunately with libpcap, there is no sure way to time things. There is the pcap_dispatch call which allows a timeout, but only on certain platforms and the implementation and behavior is also platform dependent.

Loop infinite, uses pcap_dispatch (in a loop) to limit capture to certain amount of packets.

In java, the only way for sure, is to externally break the loop (kill the thread) which is not very graceful. Libpcap mailing list is full of these types of conversations and there isn't much we can do from jnetpcap perspective. Its a limitation of libpcap, unfortunately.

The approach you took, is probably your best bet, but I would use Pcap.dispatch (in a loop) and not pcap.loop method.

Sly Technologies, Inc.
Main website: http://slytechs.com
Customer portal: http://support.slytechs.com

Come see Sly Technologies at
Ethernet Technology Summit in Santa Clara, Ca
Booth #309 April 29 - May 1, 2014

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.