flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Accola" <macc...@us.ibm.com>
Subject Using native library in Flink
Date Mon, 17 Jul 2017 19:30:09 GMT
I am new Flink user just trying to learn a little bit.  I am trying to 
incorporate an existing C++ library into a new Flink application.  I am 
seeing some strange behavior when trying to link in the native (C++) 
library using java via JNI.
I am running this on Linux (RHEL6)
I can run my application once without error.  Sometimes it will run 
successfully a 2nd or 3rd time.  However, eventually on a subsequent run, 
I get an exception about the the native library not being found:
java.lang.UnsatisfiedLinkError: no dummy2native in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        at java.lang.System.loadLibrary(System.java:1122)
        at com.att.flink.tdata.spss.TinyLoader.loadNative(Dummy2.java:10)
For debugging purposes for now, my native library does not have any 
external references.  It really contains 1 method that essentially does 
The behavior seems to indicate that there is some kind of cleanup being 
done that "unloads" the native library.  I suspect this is somehow related 
to Flink's implementation of its library cache manager, but I have not 
been able to prove this yet.
A few more details:
- I have a c++ library libdummy2native.so that contains a method that can 
be invoked via JNI.
- I have a jar containing a class, called Dummy2.  The Dummy2 constructor 
will invoke the JNI method.
- The libdummy2native.so library is invoked with System.loadLibrary() like 
     static {System.loadLibrary("dummy2native"); }
- In my simple Flink application, I have extended the ProcessFunction 
class.  Within this class, I have overriden processElement method that 
declares a Dummy2 object.
- The Dummy2 class can be called and invoked without error when used in a 
standalone java program.
Any thoughts or ideas on what to try next would be appreciated. Initially, 
I'd be happy to be able to just explain this behavior.  I will worry about 
fixing it afterwards.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message