hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: lzo error while running mr job
Date Tue, 27 Oct 2015 06:38:51 GMT
Every codec in the io.compression.codecs list of classes will be
initialised, regardless of actual further use. Since the Lzo*Codec classes
require the native library to initialise, the failure is therefore expected.

On Tue, Oct 27, 2015 at 11:42 AM Kiru Pakkirisamy <kirupakkirisamy@yahoo.com>
wrote:

> I am seeing a weird error after we moved to the new hadoop mapreduce java
> packages in 2.4
> We are not using lzo (as in io.compression.codecs) but we still get this
> error. Does it mean we have to have lzo installed even though we are not
> using ? Thanks.
>
> Regards,
> - kiru
>
> 2015-10-27 00:18:57,994 ERROR
> com.hadoop.compression.lzo.GPLNativeCodeLoader | Could not load native gpl
> library
> java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886) ~[?:1.7.0_85]
> at java.lang.Runtime.loadLibrary0(Runtime.java:849) ~[?:1.7.0_85]
> at java.lang.System.loadLibrary(System.java:1088) ~[?:1.7.0_85]
> at
> com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:31)
> [flow-trunk.242-470787.jar:?]
> at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:60)
> [flow-trunk.242-470787.jar:?]
> at java.lang.Class.forName0(Native Method) [?:1.7.0_85]
> at java.lang.Class.forName(Class.java:278) [?:1.7.0_85]
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1834)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1799)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.isSplitable(CombineFileInputFormat.java:159)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:283)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:243)
> [flow-trunk.242-470787.jar:?]
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
> [flow-trunk.242-470787.jar:?]
>
>
> Regards,
>  - kiru
>

Mime
View raw message