Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 179091039A for ; Tue, 10 Dec 2013 10:39:13 +0000 (UTC) Received: (qmail 4403 invoked by uid 500); 10 Dec 2013 10:39:07 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 4153 invoked by uid 500); 10 Dec 2013 10:39:07 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 4143 invoked by uid 99); 10 Dec 2013 10:39:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Dec 2013 10:39:06 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_FONT_SIZE_LARGE,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dwivedishashwat@gmail.com designates 209.85.215.48 as permitted sender) Received: from [209.85.215.48] (HELO mail-la0-f48.google.com) (209.85.215.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Dec 2013 10:39:02 +0000 Received: by mail-la0-f48.google.com with SMTP id n7so2464806lam.7 for ; Tue, 10 Dec 2013 02:38:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=KN7jjU7yTnHQleXI0ZGtwqs0GJ4Sz3HQxMXfO9vqTh8=; b=A7OChdxYg2T93ZexZFhxwOlDSoRSMHnnrhuwvP6GEnRwJoDTF2CZRAtPYxjgGvccli N1hf/pEz7OslTUOMXxsvEjpmCK7TsNcY5WrTOe+GnSvZVM+pSAda2sSFnAbdH6VKkDja qNwjagufQ5hlKpz7YPcnJadvQjk4VBOMNMTKapa3goOmoL0qjspy3+2kI4SrJ066fWM3 A+uGyWDJF1aCXPw0FF7x2wmUJEJMxDxq2GQJkpKqbG2aUuutZmH35Tz+1mowAw0dB6uI A9BgUMc5CI8a9suaRX1xp7bjYBxhUDf+9AmjHra1ycnWkiLMXE4Y8xjREtU4+M3NjGW0 MR/A== X-Received: by 10.152.29.130 with SMTP id k2mr257966lah.84.1386671921021; Tue, 10 Dec 2013 02:38:41 -0800 (PST) MIME-Version: 1.0 Received: by 10.114.18.162 with HTTP; Tue, 10 Dec 2013 02:38:00 -0800 (PST) In-Reply-To: <5DF48A23D7B14649BBA72C2F64C6663B82B354D1@szxeml523-mbx.china.huawei.com> References: <5DF48A23D7B14649BBA72C2F64C6663B82B354D1@szxeml523-mbx.china.huawei.com> From: shashwat shriparv Date: Tue, 10 Dec 2013 16:08:00 +0530 Message-ID: Subject: Re: Compression LZO class not found issue in Hadoop-2.2.0 To: user@hadoop.apache.org Cc: jayamviswanathan@gmail.com Content-Type: multipart/alternative; boundary=089e0158c48845c74204ed2bb76b X-Virus-Checked: Checked by ClamAV on apache.org --089e0158c48845c74204ed2bb76b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Set the class path to where hadoop*Lzo jar file is, and then try.. *Thanks & Regards * =E2=88=9E Shashwat Shriparv On Tue, Dec 10, 2013 at 4:00 PM, Vinayakumar B wr= ote: > Hi Viswa, > > > > Sorry for the late reply, > > > > Have you restarted NodeManagers after copying the lzo jars to lib? > > > > Thanks and Regards, > > Vinayakumar B > > *From:* Viswanathan J [mailto:jayamviswanathan@gmail.com] > *Sent:* 06 December 2013 23:32 > *To:* user@hadoop.apache.org > *Subject:* Compression LZO class not found issue in Hadoop-2.2.0 > > > > Hi Team, > > Have added the property in mapred/core site xml and copied the hadoop lzo > jar in hadoop lib folder. Also installed lzop,lzo-devel package in CentOS > version. > > Getting the below LZO issue in Hadoop-2.2.0, > > 3 AttemptID:attempt_1386352289515_0001_m_000000_0 Info:Error: > java.lang.IllegalArgumentException: Compression codec > com.hadoop.compression.lzo.LzoCodec was not found. > 4 at > org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java= :796) > 5 at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1000) > 6 at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:390) > 7 at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:79) > 8 at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:6= 74) > 9 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:746) > 10 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) > 11 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) > 12 at java.security.AccessController.doPrivileged(Native Method) > 13 at javax.security.auth.Subject.doAs(Subject.java:396) > 14 at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation= .java:1491) > 15 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) > 16 Caused by: java.lang.ClassNotFoundException: Class > com.hadoop.compression.lzo.LzoCodec not found > 17 at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:16= 26) > 18 at > org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java= :794) > 19 ... 11 more > > > > Do I need to add any other jars or property for this version? > > Please help. It will be very useful for production push. > > -- > Regards, > Viswa.J > --089e0158c48845c74204ed2bb76b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Set the class path to where hadoop*Lzo jar file is, and th= en try..

Thanks & Regards =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20 =09 =09 =09 =09

<= font size=3D"6">=E2=88=9E

Shash= wat Shriparv



On Tue, Dec 10, 2013 at 4:00 PM, Vinayak= umar B <vinayakumar.b@huawei.com> wrote:

Hi Viswa,

=C2=A0

Sorry for the late reply,=

=C2=A0

Have you restarted NodeMa= nagers after copying the lzo jars to lib?

=C2=A0

Thanks and Regards,

Vinayakumar B

From: Viswanat= han J [mailto:jayamviswanathan@gmail.com]
Sent: 06 December 2013 23:32
To: user= @hadoop.apache.org
Subject: Compression LZO class not found issue in Hadoop-2.2.0

=C2=A0

Hi Team,

Have added the property in mapred/core site xml and copied the hadoop lzo j= ar in hadoop lib folder. Also installed lzop,lzo-devel package in CentOS ve= rsion.

Getting the below LZO issue in Hadoop-2.2.0,

=C2=A03 AttemptID:attempt_1386352289515_0001_m_000000_0 Info:Error: java.la= ng.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.L= zoCodec was not found.
=C2=A0 4=C2=A0=C2=A0 at org.apache.hadoop.mapred.JobConf.getMapOutputCompre= ssorClass(JobConf.java:796)
=C2=A0 5=C2=A0=C2=A0 at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.in= it(MapTask.java:1000)
=C2=A0 6=C2=A0=C2=A0 at org.apache.hadoop.mapred.MapTask.createSortingColle= ctor(MapTask.java:390)
=C2=A0 7=C2=A0=C2=A0 at org.apache.hadoop.mapred.MapTask.access$100(MapTask= .java:79)
=C2=A0 8=C2=A0=C2=A0 at org.apache.hadoop.mapred.MapTask$NewOutputCollector= .<init>(MapTask.java:674)
=C2=A0 9=C2=A0=C2=A0 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTa= sk.java:746)
=C2=A010=C2=A0=C2=A0 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:3= 39)
=C2=A011=C2=A0=C2=A0 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.= java:162)
=C2=A012=C2=A0=C2=A0 at java.security.AccessController.doPrivileged(Native = Method)
=C2=A013=C2=A0=C2=A0 at javax.security.auth.Subject.doAs(Subject.java:396)<= br> =C2=A014=C2=A0=C2=A0 at org.apache.hadoop.security.UserGroupInformation.doA= s(UserGroupInformation.java:1491)
=C2=A015=C2=A0=C2=A0 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.j= ava:157)
=C2=A016 Caused by: java.lang.ClassNotFoundException: Class com.hadoop.comp= ression.lzo.LzoCodec not found
=C2=A017=C2=A0=C2=A0 at org.apache.hadoop.conf.Configuration.getClassByName= (Configuration.java:1626)
=C2=A018=C2=A0=C2=A0 at org.apache.hadoop.mapred.JobConf.getMapOutputCompre= ssorClass(JobConf.java:794)
=C2=A019=C2=A0=C2=A0 ... 11 more

=C2=A0

Do I need to add any other jars or property for this= version?

Please help. It will be very useful for production push.

--
Regards,
Viswa.J


--089e0158c48845c74204ed2bb76b--