hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: LZO and native hadoop libraries
Date Wed, 01 Oct 2008 20:34:44 GMT

On Oct 1, 2008, at 12:54 PM, Nathan Marz wrote:

> Yes, this is exactly what I'm seeing. To be honest, I don't know  
> which LZO native library it should be looking for. The LZO install  
> dropped "liblzo2.la" and "liblzo2.a" in my /usr/local/lib directory,  
> but not a file with a ".so" extension. Hardcoding would be fine as a  
> temporary solution, but I don't know what to hardcode.
>

You do need liblzo2.so for this to work.

The hardcoded value has to be liblzo2.so too ...

Arun

> Thanks,
> Nathan
>
>
> On Sep 30, 2008, at 8:45 PM, Amareshwari Sriramadasu wrote:
>
>> Are you seeing HADOOP-2009?
>>
>> Thanks
>> Amareshwari
>> Nathan Marz wrote:
>>> Unfortunately, setting those environment variables did not help my  
>>> issue. It appears that the "HADOOP_LZO_LIBRARY" variable is not  
>>> defined in both LzoCompressor.c and LzoDecompressor.c. Where is  
>>> this variable supposed to be set?
>>>
>>>
>>>
>>> On Sep 30, 2008, at 12:33 PM, Colin Evans wrote:
>>>
>>>> Hi Nathan,
>>>> You probably need to add the Java headers to your build path as  
>>>> well - I don't know why the Mac doesn't ship with this as a  
>>>> default setting:
>>>>
>>>> export CPATH="/System/Library/Frameworks/JavaVM.framework/ 
>>>> Versions/CurrentJDK/Home/include "
>>>> export CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/ 
>>>> Versions/CurrentJDK/Home/include"
>>>>
>>>>
>>>>
>>>>
>>>> Nathan Marz wrote:
>>>>> Thanks for the help. I was able to get past my previous issue,  
>>>>> but the native build is still failing. Here is the end of the  
>>>>> log output:
>>>>>
>>>>>   [exec]     then mv -f ".deps/LzoCompressor.Tpo" ".deps/ 
>>>>> LzoCompressor.Plo"; else rm -f ".deps/LzoCompressor.Tpo"; exit  
>>>>> 1; fi
>>>>>   [exec] mkdir .libs
>>>>>   [exec]  gcc -DHAVE_CONFIG_H -I. -I/Users/nathan/Downloads/ 
>>>>> hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo - 
>>>>> I../../../../../../.. -I/Library/Java/Home//include -I/Users/ 
>>>>> nathan/Downloads/hadoop-0.18.1/src/native/src -g -Wall -fPIC -O2  
>>>>> -m32 -g -O2 -MT LzoCompressor.lo -MD -MP -MF .deps/ 
>>>>> LzoCompressor.Tpo -c /Users/nathan/Downloads/hadoop-0.18.1/src/ 
>>>>> native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c  - 
>>>>> fno-common -DPIC -o .libs/LzoCompressor.o
>>>>>   [exec] /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/ 
>>>>> org/apache/hadoop/io/compress/lzo/LzoCompressor.c: In function  
>>>>> 'Java_org_apache_hadoop_io_compress_lzo_LzoCompressor_initIDs':
>>>>>   [exec] /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/ 
>>>>> org/apache/hadoop/io/compress/lzo/LzoCompressor.c:135: error:  
>>>>> syntax error before ',' token
>>>>>   [exec] make[2]: *** [LzoCompressor.lo] Error 1
>>>>>   [exec] make[1]: *** [all-recursive] Error 1
>>>>>   [exec] make: *** [all] Error 2
>>>>>
>>>>>
>>>>> Any ideas?
>>>>>
>>>>>
>>>>>
>>>>> On Sep 30, 2008, at 11:53 AM, Colin Evans wrote:
>>>>>
>>>>>> There's a patch to get the native targets to build on Mac OS X:
>>>>>>
>>>>>> http://issues.apache.org/jira/browse/HADOOP-3659
>>>>>>
>>>>>> You probably will need to monkey with LDFLAGS as well to get it 

>>>>>> to work, but we've been able to build the native libs for the  
>>>>>> Mac without too much trouble.
>>>>>>
>>>>>>
>>>>>> Doug Cutting wrote:
>>>>>>> Arun C Murthy wrote:
>>>>>>>> You need to add libhadoop.so to your java.library.patch.
 
>>>>>>>> libhadoop.so is available in the corresponding release in
the  
>>>>>>>> lib/native directory.
>>>>>>>
>>>>>>> I think he needs to first build libhadoop.so, since he appears
 
>>>>>>> to be running on OS X and we only provide Linux builds of this
 
>>>>>>> in releases.
>>>>>>>
>>>>>>> Doug
>>>>>>
>>>>>
>>>>
>>>
>>
>


Mime
View raw message