lucene-pylucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Chyla <roman.ch...@gmail.com>
Subject Re: Problem loading jcc from java : undefined symbol: PyExc_IOError
Date Tue, 15 Feb 2011 15:51:18 GMT
In the end, I compiled a new python with the necessary modules, and
that works just fine.
But it was an interesting experience. Thank you Andi, your help is always great.

Cheers,

  roman

On Tue, Feb 15, 2011 at 9:22 AM, Roman Chyla <roman.chyla@gmail.com> wrote:
> On Tue, Feb 15, 2011 at 4:22 AM, Andi Vajda <vajda@apache.org> wrote:
>>
>> On Tue, 15 Feb 2011, Roman Chyla wrote:
>>
>>> from:
>>> http://realmike.org/blog/2010/07/18/python-extensions-in-cpp-using-swig/
>>>
>>> Q. ?Fatal Python error: Interpreter not initialized (version mismatch?)?
>>>
>>> A. This error occurs when the version of the Python interpreter for
>>> which the extension module has been built is different from the
>>> version of the interpreter that attempts to import the module.
>>>
>>> Is there a way to find out which python interpreter version is inside
>>> JCC? Also, Is it somehow possible that the java process that load jcc
>>> library will be picking the default python (2.4) instead of the python
>>> (2.5)? PATH is set to python2.5.
>>
>> There is no Python interpreter inside jcc. It's dynamically linked.
>> To know which version of the shared library is looked for and expected, use
>> the 'ldd' utility against the various shared libraries involved to tell you.
>> That version is selected at build time, when you run 'python setup.py ...'
>> That version of python determines the version of libpython.so used.
>
> This will be probably the problem (as you said before), the libjcc.so
> shows no python -
>
> bash-3.2$ ldd build/lib.linux-x86_64-2.5/libjcc.so
>        linux-vdso.so.1 =>  (0x00007fff7affc000)
>        /$LIB/snoopy.so => /lib64/snoopy.so (0x00002b8ed0e74000)
>        libjava.so => /afs/cern.ch/user/r/rchyla/public/jdk1.6.0_18/jre/lib/amd64/libjava.so
> (0x00002b8ed1076000)
>        libjvm.so => /afs/cern.ch/user/r/rchyla/public/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
> (0x00002b8ed11a5000)
>        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002b8ed1c3f000)
>        libm.so.6 => /lib64/libm.so.6 (0x00002b8ed1f3f000)
>        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b8ed21c2000)
>        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b8ed23cf000)
>        libc.so.6 => /lib64/libc.so.6 (0x00002b8ed25eb000)
>        libdl.so.2 => /lib64/libdl.so.2 (0x00002b8ed2943000)
>        libverify.so =>
> /afs/cern.ch/user/r/rchyla/public/jdk1.6.0_18/jre/lib/amd64/libverify.so
> (0x00002b8ed2b47000)
>        libnsl.so.1 => /lib64/libnsl.so.1 (0x00002b8ed2c57000)
>        /lib64/ld-linux-x86-64.so.2 (0x00002b8ed08c9000)
>
> And I think, the python2.4 (the default  on the system) is being
> loaded -- but how to force loading of python2.5 (if that was possible
> at all) I don't know. Compilation is definitely done with -lpython2.5
>
> Cheers,
>
>  roman
>
>>
>> Andi..
>>
>>>
>>> Cheers,
>>>
>>>  roman
>>>
>>>
>>> On Tue, Feb 15, 2011 at 2:40 AM, Roman Chyla <roman.chyla@gmail.com>
>>> wrote:
>>>>
>>>> On Tue, Feb 15, 2011 at 1:32 AM, Andi Vajda <vajda@apache.org> wrote:
>>>>>
>>>>> On Tue, 15 Feb 2011, Roman Chyla wrote:
>>>>>
>>>>>> The python embedded in Java works really well on MacOsX and also
>>>>>> Ubuntu. But I am trying hard to make it work also on Scientific Linux
>>>>>> (SLC5) with *statically* built Python. The python is a build from
>>>>>> ActiveState.
>>>>>
>>>>> You mean you're going to try to dynamically load libpython.a into a JVM
>>>>> ?
>>>>> I have no idea if this can work at all.
>>>>
>>>> I am very ignorant as far as the difference between statically and
>>>> dynamically linked libraries go - I just wanted to use JCC wrapped
>>>> code with this particular statically linked python
>>>>
>>>> I got little bit further, but just little:
>>>>
>>>> after I changed -Xlinker --export-dynamic into "-Xlinker
>>>> -export-dynamic" (and installed python into /opt...) I am getting a
>>>> different error:
>>>>
>>>> SEVERE: org.apache.jcc.PythonException: No module named
>>>> solrpie.java_bridge
>>>> null
>>>>        at org.apache.jcc.PythonVM.instantiate(Native Method)
>>>>        at rca.python.jni.PythonVMBridge.start(Unknown Source)
>>>>        at rca.python.jni.PythonVMBridge.start(Unknown Source)
>>>>        at rca.python.jni.PythonVMBridge.start(Unknown Source)
>>>>        at rca.python.jni.SolrpieVM.getBridge(Unknown Source)
>>>>
>>>>
>>>> My understanding is that the previous error has gone (and the python
>>>> module "time" is loaded), because if I set PYTHONPATH incorrectly, I
>>>> get:
>>>> This message is IMHO coming from Python
>>>>
>>>> But when I correct the PYTHONPATH, I am getting only this:
>>>>
>>>> [java] Fatal Python error: Interpreter not initialized (version
>>>> mismatch?)
>>>> [java] Java Result: 134
>>>>
>>>>
>>>>
>>>>>
>>>>> If my understanding of static builds is correct, I'd imagine the only
>>>>> way
>>>>> for this to work would be to statically compile the JVM (hotspot) and
>>>>> python
>>>>> together.
>>>>
>>>> oooups, that is way over my head
>>>>
>>>>>
>>>>> But why all this ?
>>>>
>>>> Because on the grid, we already had a statically linked python and it
>>>> was working very well with pylucene (and after all, I managed to make
>>>> it work also for solr and other packages)
>>>>
>>>> But if you think that it is not possible, I should do something else :)
>>>> But it was fun trying, if you get some idea, please let me know.
>>>>
>>>> Thank you,
>>>>
>>>>  Roman
>>>>
>>>>>
>>>>> Andi..
>>>>>
>>>>>> So far, I managed to build all the needed extensions (jcc, lucene,
>>>>>> solr) and I can run them in python, but when I try to start the java
>>>>>> app and use python, I get:
>>>>>>
>>>>>> SEVERE: org.apache.jcc.PythonException:
>>>>>>
>>>>>>
>>>>>> /afs/cern.ch/user/r/rchyla/public/ActivePython-2.5.5.7-linux-x86_64/INSTALLDIR/lib/python2.5/lib-dynload/time.so:
>>>>>> undefined symbol: PyExc_IOError
>>>>>>
>>>>>>
>>>>>> I understand, that the missing symbol PyExc_IOError is in the static
>>>>>> python library:
>>>>>>
>>>>>> bash-3.2$ nm
>>>>>>
>>>>>> /afs/cern.ch/user/r/rchyla/public/ActivePython-2.5.5.7-linux-x86_64/INSTALLDIR/lib/python2.5/config/libpython2.5.a
>>>>>> | grep IOError
>>>>>> 0000000000004120 D PyExc_IOError
>>>>>> 0000000000004140 d _PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>                U PyExc_IOError
>>>>>>
>>>>>> So when building JCC, I build with these arguments:
>>>>>>
>>>>>> lflags  +  ['-lpython%s.%s' %(sys.version_info[0:2]),
>>>>>> '-L',
>>>>>>
>>>>>>
>>>>>> '/afs/cern.ch/user/r/rchyla/public/ActivePython-2.5.5.7-linux-x86_64/INSTALLDIR/lib/python2.5/config',
>>>>>> '-rdynamic',
>>>>>> '-Wl,--export-dynamic',
>>>>>> '-Xlinker',
>>>>>> '--export-dynamic']
>>>>>>
>>>>>> I just found instructions at:
>>>>>>
>>>>>>
>>>>>> http://stackoverflow.com/questions/4223312/python-interpreter-embedded-in-the-application-fails-to-load-native-modules
>>>>>> I don't really understand g++, but the symbol is there after the
>>>>>> compilation
>>>>>>
>>>>>> bash-3.2$ nm
>>>>>>
>>>>>> /afs/cern.ch/user/r/rchyla/public/ActivePython-2.5.5.7-linux-x86_64/INSTALLDIR/lib/python2.5/site-packages/JCC-2.7-py2.5-linux-x86_64.egg/libjcc.so
>>>>>> | grep IOError
>>>>>> 0000000000352240 D PyExc_IOError
>>>>>> 0000000000352260 d _PyExc_IOError
>>>>>>
>>>>>> And when starting java, I do
>>>>>>
>>>>>>
>>>>>> "-Djava.library.path=/afs/cern.ch/user/r/rchyla/public/ActivePython-2.5.5.7-linux-x86_64/INSTALLDIR/lib/python2.5/site-packages/JCC-2.7-py2.5-linux-x86_64.egg"
>>>>>>
>>>>>> The code works find on mac (python 2.6) and ubuntu (python2.6), but
>>>>>> not this statically linked python2.5 - would you know what I can
try?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>>
>>>>>>  roman
>>>>>>
>>>>>>
>>>>>> PS: I tried several compilations, but I was usually re-compiling
JCC
>>>>>> without building lucene etc again, I hope that is not the problem.
>>>>>>
>>>>>
>>>>
>>
>

Mime
View raw message