Loading libraries and some dynamic flexibility

2 replies [Last post]
Mark Bednarczyk
Offline
Joined: 03/22/2008

I have been researching and working on various library loading related issues. Currently jnetpcap loads the top level system library the "jnetpcap" library. On windows systems that translates to jnetpcap.dll and on other systems to either libpcap.so or libpcap.so.MAROR.MINOR versions. My build environment builds these shared libraries using standard libpcap library installation as a prerequisite and supplied on the linker command line.

This works in most cases but not all. As a library builder and distributor, I have no control over user's environment. There could be any version of libpcap on user's system. On windows, this is rather straight forward. On *NUX systems its up the installed packages, custom libpcap builds and versions of those libraries installed.

The biggest problem I have ran into is that jnetpcap built on ubuntu system will have different dependencies then on a fedora-core system. The problem is that the compile time linker creates a 'DT_NEEDED' section with the name it finds in the 'DT_SONAME' of the libpcap it happens to use at compile time. Unfortunately, libpcap developers chose not to leave the 'DT_SONAME' field as a generic 'libpcap.so' but something extremely specific like 'libpcap.so.0.8' or 'libpcap.so.0.9' From then on, the final 'libjnetpcap.so' file is always and forever dependent on that specific version of the library. These dependencies are different based on the system and package installed.

Instead of relying on the dynamic linker to make the best choice with the available installed package on the user's system, it will instead look for a very narrow version of 'libpcap.so.MAJOR.MINOR' version of the library.

Here is a specific examples.

Ubuntu system:

ubuntu9:~/prjs/jnp_1.4j> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 9.04
Release:        9.04
Codename:       jaunty

ubuntu9:~/prjs/jnp_1.4j> ls -l /usr/lib64/libpcap*
-rw-r--r-- 1 root root 352372 2009-03-31 13:12 /usr/lib64/libpcap.a
lrwxrwxrwx 1 root root     14 2011-01-18 14:10 /usr/lib64/libpcap.so -> libpcap.so.0.8
lrwxrwxrwx 1 root root     16 2011-01-18 14:04 /usr/lib64/libpcap.so.0.8 -> libpcap.so.1.0.0
-rw-r--r-- 1 root root 212912 2009-03-31 13:12 /usr/lib64/libpcap.so.1.0.0

ubuntu9:~/prjs/jnp_1.4j> readelf -d build/lib/libjnetpcap.so

Dynamic section at offset 0x26968 contains 24 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpcap.so.0.8]
 0x000000000000000e (SONAME)             Library soname: [libjnetpcap.so]

And similarily configured fedora-core:

[mark@b2-fc10x jnp_1.4j]$ lsb_release -a
LSB Version:    :core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:desktop-3.1-amd64:desktop-3.1-noarch:desktop-3.2-amd64:desktop-3.2-noarch
Distributor ID: Fedora
Description:    Fedora release 10 (Cambridge)
Release:        10
Codename:       Cambridge
[mark@b2-fc10x jnp_1.4j]$ ls -l /usr/lib64/libpcap*
lrwxrwxrwx 1 root root     14 2010-08-21 16:37 /usr/lib64/libpcap.so -> libpcap.so.0.9
lrwxrwxrwx 1 root root     16 2010-08-21 10:01 /usr/lib64/libpcap.so.0.9 -> libpcap.so.0.9.8
-rwxr-xr-x 1 root root 180528 2008-06-27 08:27 /usr/lib64/libpcap.so.0.9.8
[mark@b2-fc10x jnp_1.4j]$ readelf -d build/lib/libjnetpcap.so

Dynamic section at offset 0x24af0 contains 23 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpcap.so.0.9]
 0x000000000000000e (SONAME)             Library soname: [libjnetpcap.so]

The key lines are the (NEEDED) libpcap.so.0.8 and libpcap.so.0.9, which is where all the trouble for most users ends up being. Since there is no way to override this at jnetpcap link time and put it our own values using any linker options, there are 3 alternatives.

1) Do nothing and either provide multiple versions of libjnetpcap.so with different DT_NEED dependencies for libpcap.

2) Build libpcap myself on and override the DT_SONAME parameter and specify the generic libpcap.so as the name

3) Make a copy of the default libpcap.so file to our build directory and do a binary replace on the DT_NEEDED field and rewrite the name. This is doable because under all circumstances we are always shortening the name and thus can do inplace binary substitution on the binary file.

Here is what the results look like when I tried #3:


ubuntu9:~/prjs/jnp_1.4j> readelf -d build/lib/libpcap.so build/lib/libjnetpcap.so

File: build/lib/libpcap.so

Dynamic section at offset 0x32d40 contains 22 entries:
  Tag        Type                         Name/Value
 0x000000000000000e (SONAME)             Library soname: [libpcap.so.0.8]

File: build/lib/libjnetpcap.so

Dynamic section at offset 0x26968 contains 24 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpcap.so.0.8]
 0x000000000000000e (SONAME)             Library soname: [libjnetpcap.so]

ubuntu9:~/prjs/jnp_1.4j> ldd build/lib/libjnetpcap.so
        libpcap.so.0.8 => /usr/lib/libpcap.so.0.8 (0x00007fec0d0c5000)

And here is after we do a substitution and get rid of the versioned libpcap:

ubuntu9:~/prjs/jnp_1.4j> perl -pi -e 's/libpcap.so..../libpcap.so\x0\x0\x0\x0/g' build/lib/libjnetpcap.so

ubuntu9:~/prjs/jnp_1.4j> readelf -d build/lib/libpcap.so build/lib/libjnetpcap.so
File: build/lib/libpcap.so

Dynamic section at offset 0x32d40 contains 22 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libpcap.so.0.8]
File: build/lib/libjnetpcap.so

Dynamic section at offset 0x26968 contains 24 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpcap.so]
 0x000000000000000e (SONAME)             Library soname: [libjnetpcap.so]
 
ubuntu9:~/prjs/jnp_1.4j> ldd build/lib/libjnetpcap.so
        libpcap.so => build/lib/libpcap.so (0x00007f0f4b812000)

ubuntu9:~/prjs/jnp_1.4j> unsetenv LD_LIBRARY_PATH

ubuntu9:~/prjs/jnp_1.4j> ldd build/lib/libjnetpcap.so
        libpcap.so => /usr/lib/libpcap.so (0x00007fea0a7e8000)

As you can see that jnetpcap now nicely resolves to the unversioned 'libpcap.so' wherever its found. At first we had LD_LIBRARY_PATH set to point at build/lib directory where we have a copy of 'libpcap.so' file we just modified. When we unset this variable and default to system search path, we now point at '/usr/lib/libpcap.so'. This is exactly what we need.

I have tested this and it seems to work correctly. If anyone has any feedback or better way of accomplishing this, please let me know.

Sly Technologies, Inc.
http://slytechs.com

timothy.godfrey
Offline
Joined: 03/13/2011
dynamic library loading

Hi,

I think I am having the same problem loading the correct libpcap shared object. I am using jnetpcap-1.3.b4 and its version of libjnetpcap.so depends on libpcap.so.0.9, but I have libpcap.so.0.8.

I have read and understood your description of the problem above, but I do not understand how you are changing the dependency of the libjnetpcap.so to depend on a generic libpcap.so.

When I attempt to use the jnetpcap api, the compiler error I get is:

Exception in thread "main" java.lang.UnsatisfiedLinkError: [my/installation/path]/jnetpcap-1.3.b4/libjnetpcap.so: libpcap.so.0.9: cannot open shared object file: No such file or directory.

Thanks for the help,
Tim

Mark Bednarczyk
Offline
Joined: 03/22/2008
The dependency change

The dependency change happened with version 1.4. Version 1.3 still has the version specific dependency on libpcap.

Sly Technologies, Inc.
http://slytechs.com

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.