hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kozlov <ale...@cloudera.com>
Subject Re: How to get lzo compression library loaded?
Date Sun, 01 Aug 2010 03:37:19 GMT
Hi Alex,

org.apache.hadoop.io.compress.GzipCodec is in the hadoop-core-*.jar.  You
don't need to add /usr/local/hadoop/hadoop-0.20.2/lib/ to the classpath
since hadoop shell script does it for you.  What is exactly the command you
are trying to run?  Can you also give the output of 'env | grep -i hadoop'
and 'env | grep -i classpath'.

Alex K


On Sat, Jul 31, 2010 at 1:38 AM, Alex Luya <alexander.luya@gmail.com> wrote:

> Hello:
>    I have followed this link:http://code.google.com/p/hadoop-gpl-
> compression/wiki/FAQ<http://code.google.com/p/hadoop-gpl-%0Acompression/wiki/FAQ>to
install lzo compression library,and copy
> hadoop-lzo-0.4.4.jar to $HADOOP_HOME/lib,and all files under
> ..lib/native/Linux-amd64-64 to $HADOOP_HOME/lib/native/Linux-amd64-64,
> and run example,but got this errors:
>
> ----------------------------------------------------------------------------------------------------------------
> Exception in thread "main" java.lang.IllegalArgumentException: Compression
> codec
>                org.apache.hadoop.io.compress.GzipCodec not found.
>        at
>
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
>        at
>
> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
>        at
> com.hadoop.compression.lzo.LzoIndex.createIndex(LzoIndex.java:202)
>        at
> com.hadoop.compression.lzo.LzoIndexer.indexSingleFile(LzoIndexer.java:117)
>        at
> com.hadoop.compression.lzo.LzoIndexer.indexInternal(LzoIndexer.java:98)
>        at com.hadoop.compression.lzo.LzoIndexer.index(LzoIndexer.java:52)
>        at com.hadoop.compression.lzo.LzoIndexer.main(LzoIndexer.java:137)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.ClassNotFoundException:
>                org.apache.hadoop.io.compress.GzipCodec
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>        at java.lang.Class.forName0(Native Method)
>        at java.lang.Class.forName(Class.java:247)
>        at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
>        at
>
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
>        ... 11 more
>
>
>
> ----------------------------------------------------------------------------------------------------------------
>
> and then I try to add this to hadoop-env.sh:
>
> export
> HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/hadoop/hadoop-0.20.2/lib/
>
> Same problem as before,this problem is killing me,because it has surrounded
> me
> for one month.
>
>
>

Mime
View raw message