hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinayakumar B <vinayakuma...@huawei.com>
Subject RE: Compression LZO class not found issue in Hadoop-2.2.0
Date Tue, 10 Dec 2013 10:30:31 GMT
Hi Viswa,

Sorry for the late reply,

Have you restarted NodeManagers after copying the lzo jars to lib?

Thanks and Regards,
Vinayakumar B
From: Viswanathan J [mailto:jayamviswanathan@gmail.com]
Sent: 06 December 2013 23:32
To: user@hadoop.apache.org
Subject: Compression LZO class not found issue in Hadoop-2.2.0

Hi Team,

Have added the property in mapred/core site xml and copied the hadoop lzo jar in hadoop lib
folder. Also installed lzop,lzo-devel package in CentOS version.
Getting the below LZO issue in Hadoop-2.2.0,

 3 AttemptID:attempt_1386352289515_0001_m_000000_0 Info:Error: java.lang.IllegalArgumentException:
Compression codec com.hadoop.compression.lzo.LzoCodec was not found.
  4   at org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:796)
  5   at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1000)
  6   at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:390)
  7   at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:79)
  8   at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674)
  9   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:746)
 10   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
 11   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
 12   at java.security.AccessController.doPrivileged(Native Method)
 13   at javax.security.auth.Subject.doAs(Subject.java:396)
 14   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
 15   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
 16 Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec
not found
 17   at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
 18   at org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:794)
 19   ... 11 more

Do I need to add any other jars or property for this version?

Please help. It will be very useful for production push.


View raw message