Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DE221FDEB for ; Thu, 4 Apr 2013 04:43:43 +0000 (UTC) Received: (qmail 37477 invoked by uid 500); 4 Apr 2013 04:43:38 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 37364 invoked by uid 500); 4 Apr 2013 04:43:38 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 37338 invoked by uid 99); 4 Apr 2013 04:43:37 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Apr 2013 04:43:37 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yypvsxf19870706@gmail.com designates 209.85.220.53 as permitted sender) Received: from [209.85.220.53] (HELO mail-pa0-f53.google.com) (209.85.220.53) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Apr 2013 04:43:32 +0000 Received: by mail-pa0-f53.google.com with SMTP id bh4so1264725pad.40 for ; Wed, 03 Apr 2013 21:43:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:subject:references:from:content-type:x-mailer :in-reply-to:message-id:date:to:content-transfer-encoding :mime-version; bh=MAVRHxJd9tu5PBUiMdNNhpv+vn7Chno8j/rHhnHcCs4=; b=W5USsjwMkAC01kcG5Y1u7M5trG7timQ1mjg+QBtLKqZ/bW6EiHwbANzaF4qlZviV/j dPKSCJbaTcT/RrlAEBhEFodspQbbtEGCgzEhG/9pdrLBBGvCSImvMu09qnuDr7O5Vf79 6MbRewmAjiq2ioEMLeuwGw5j2oqT9io+4ShARt5ExvGKd88ejz/WCY9eDLy5J98ACuUC Z0jldyuaL6bFX9+vjAT9U+/Nl7spHmfBcA9yHvhUdGtT8t8wDIWrSneZdnbNpXc4QYnh QWS6uSMXycbEELezEwjtUDOTpP4MI6s0h+A2owiV6dh1MFXWuCVZaakFGWfnlXic4+41 f5pw== X-Received: by 10.66.250.161 with SMTP id zd1mr7359398pac.141.1365050592081; Wed, 03 Apr 2013 21:43:12 -0700 (PDT) Received: from [10.3.103.176] ([122.96.45.155]) by mx.google.com with ESMTPS id jb1sm8589565pbc.24.2013.04.03.21.43.08 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 03 Apr 2013 21:43:11 -0700 (PDT) Subject: Re: Error while running MapR program on multinode configuration References: <7EA0FAED-672D-4DBF-80CC-2660D7A2BCB6@gmail.com> From: yypvsxf19870706 Content-Type: multipart/alternative; boundary=Apple-Mail-9615C245-4CC1-4AC5-8195-273F3578680F X-Mailer: iPhone Mail (10B146) In-Reply-To: Message-Id: <4FB51349-2BF3-49E9-B14C-B68398C47DD5@gmail.com> Date: Thu, 4 Apr 2013 12:41:57 +0800 To: "user@hadoop.apache.org" Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail-9615C245-4CC1-4AC5-8195-273F3578680F Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable Hi Mohammad Thanks for your response . As far as I know,the jars which the client submit to the MP through the n= et in DistributedCache mode because of the jar localization . The point that I aim at is we need to reduce the size of the objects we s= ubmit through network ,which will save the bandwidth and the time to run the= job. Maybe I was wrong ,discussion will be appreciated. =B7=A2=D7=D4=CE=D2=B5=C4 iPhone =D4=DA 2013-4-4=A3=AC7:47=A3=ACMohammad Tariq =D0=B4=B5= =C0=A3=BA > DistributedCache?? >=20 > Warm Regards, > Tariq > https://mtariq.jux.com/ > cloudfront.blogspot.com >=20 >=20 > On Thu, Apr 4, 2013 at 5:01 AM, yypvsxf19870706 wrote: >> Hi all >>=20 >> However,I do think the export jar contains external libraries is too h= uge to be submitted to the MP by the client. >> So It comes to the conclusion that the jar without the external jars c= ould be suited the situation ,however the runnable jar brings the errors. >> How to solve the errors. One way is to set the Hadoop_classspath expli= citly when you run the runnable jars. >> Any other suggestions? >>=20 >> Regards >>=20 >>=20 >>=20 >> =B7=A2=D7=D4=CE=D2=B5=C4 iPhone >>=20 >> =D4=DA 2013-4-4=A3=AC6:47=A3=ACMohammad Tariq =D0=B4= =B5=C0=A3=BA >>=20 >>> As expected :) >>> Actually this is something related to Eclipse. When it comes to the crea= tion of executable jars containing external jars we need to be a bit careful= . One thing which you could probably try is to select the "Package required l= ibraries into generated jar" option while exporting your project as an execu= table jar. >>>=20 >>> HTH >>>=20 >>> Warm Regards, >>> Tariq >>> https://mtariq.jux.com/ >>> cloudfront.blogspot.com >>>=20 >>>=20 >>> On Thu, Apr 4, 2013 at 4:10 AM, Varsha Raveendran wrote: >>>> Thank you! You are right! >>>> I created the jar file using the command line and not eclipse and it w= orked! >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> On Thu, Apr 4, 2013 at 4:06 AM, Mohammad Tariq wro= te: >>>>> It seems to be a non-hadoop issue to me. Is the jar which you are fina= lly creating a runnable jar?We can't reference external classes(jars) in a r= unnable jar file. Everything must be contained inside the jar itself. >>>>>=20 >>>>> Warm Regards, >>>>> Tariq >>>>> https://mtariq.jux.com/ >>>>> cloudfront.blogspot.com >>>>>=20 >>>>>=20 >>>>> On Thu, Apr 4, 2013 at 3:28 AM, Varsha Raveendran wrote: >>>>>> Thanks for replying!=20 >>>>>>=20 >>>>>> version : hadoop 1.1.1 >>>>>>=20 >>>>>> I am creating a jar file called ga_test.jar and then placing it in t= he build path of another program.=20 >>>>>> Also, I am using eclipse to create another jar file including ga_test= .jar as a reference library. I do not know why hadoop is taking the filename= as rsrc:ga_test.jar. (Maybe thats where the ":" is coming into the picture= !) >>>>>>=20 >>>>>> Thanks! >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> On Thu, Apr 4, 2013 at 3:18 AM, Mohammad Tariq w= rote: >>>>>>> Which version are you using? Could you plz show me your code, if pos= sible for you.=20 >>>>>>> Also, how are you running the job? For a detailed explanation you mi= ght find this useful. >>>>>>>=20 >>>>>>> Warm Regards, >>>>>>> Tariq >>>>>>> https://mtariq.jux.com/ >>>>>>> cloudfront.blogspot.com >>>>>>>=20 >>>>>>>=20 >>>>>>> On Thu, Apr 4, 2013 at 3:09 AM, Varsha Raveendran wrote: >>>>>>>> But I do not have a ":" in any of my file names.=20 >>>>>>>> What could the other reasons be? I am not able to debug the error..= >>>>>>>>=20 >>>>>>>> Thank you for replying. >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> On Thu, Apr 4, 2013 at 2:02 AM, Mohammad Tariq = wrote: >>>>>>>>> Hello ma'am, >>>>>>>>>=20 >>>>>>>>> Please make sure that you don't have ":" in your files. >>>>>>>>>=20 >>>>>>>>> Warm Regards, >>>>>>>>> Tariq >>>>>>>>> https://mtariq.jux.com/ >>>>>>>>> cloudfront.blogspot.com >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On Thu, Apr 4, 2013 at 1:50 AM, Varsha Raveendran wrote: >>>>>>>>>> Hello, >>>>>>>>>>=20 >>>>>>>>>> I am facing this error while trying to run a jar file on hadoop := >>>>>>>>>>=20 >>>>>>>>>> 13/04/04 01:48:01 INFO mapred.JobClient: Cleaning up the staging a= rea hdfs://MT2012158:54310/app/hadoop/tmp/mapred/staging/hduser/.staging/job= _201304032344_0008 >>>>>>>>>> Exception in thread "main" java.lang.reflect.InvocationTargetExce= ption >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method= ) >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAc= cessorImpl.java:57) >>>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegating= MethodAccessorImpl.java:43) >>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:616) >>>>>>>>>> at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main= (JarRsrcLoader.java:58) >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method= ) >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAc= cessorImpl.java:57) >>>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegating= MethodAccessorImpl.java:43) >>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:616) >>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>>>>>>> Caused by: java.lang.IllegalArgumentException: java.net.URISyntax= Exception: Relative path in absolute URI: rsrc:ga_test.jar >>>>>>>>>> at org.apache.hadoop.fs.Path.initialize(Path.java:148) >>>>>>>>>> at org.apache.hadoop.fs.Path.(Path.java:126) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(J= obClient.java:841) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(J= obClient.java:717) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient.access$400(JobClient.ja= va:179) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:92= 7) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:91= 2) >>>>>>>>>> at java.security.AccessController.doPrivileged(Native Method)= >>>>>>>>>> at javax.security.auth.Subject.doAs(Subject.java:416) >>>>>>>>>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserG= roupInformation.java:1136) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobCl= ient.java:912) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.jav= a:886) >>>>>>>>>> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1= 323) >>>>>>>>>> at edu.iiitb.config.GAMain.launch(Unknown Source) >>>>>>>>>> at edu.iiitb.config.GAMain.run(Unknown Source) >>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>>>>>>>> at edu.iiitb.config.GAMain.runGA(Unknown Source) >>>>>>>>>> at TC1.main(TC1.java:21) >>>>>>>>>> ... 10 more >>>>>>>>>> Caused by: java.net.URISyntaxException: Relative path in absolute= URI: rsrc:ga_test.jar >>>>>>>>>> at java.net.URI.checkPath(URI.java:1804) >>>>>>>>>> at java.net.URI.(URI.java:752) >>>>>>>>>> at org.apache.hadoop.fs.Path.initialize(Path.java:145) >>>>>>>>>> ... 27 more >>>>>>>>>>=20 >>>>>>>>>> Please help!!! >>>>>>>>>>=20 >>>>>>>>>> --=20 >>>>>>>>>> -Varsha=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> --=20 >>>>>>>> -Varsha=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> --=20 >>>>>> -Varsha=20 >>>>=20 >>>>=20 >>>>=20 >>>> --=20 >>>> -Varsha=20 >=20 --Apple-Mail-9615C245-4CC1-4AC5-8195-273F3578680F Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi Mohammad

&= nbsp;  Thanks for your response .

   = ; As far as I know,the jars which the client submit to the MP through the ne= t in DistributedCache m= ode because of the jar localization .
   The point that I aim at is we need to= reduce the size of the objects we submit through network ,which will save t= he bandwidth and the time to run the job.
   Maybe I was wrong ,discussion wil= l be appreciated.

=E5=8F=91=E8=87=AA=E6=88=91=E7=9A=84 i= Phone

=E5=9C=A8 2013-4-4=EF=BC=8C7:47=EF=BC=8CMohammad Tariq &= lt;dontariq@gmail.com> =E5=86=99= =E9=81=93=EF=BC=9A

DistributedCache??



On Thu, Apr 4, 2013 at 5:01 AM, yypvsxf19= 870706 <yypvsxf19870706@gmail.com> wrote:
Hi all

   However,= I do think the export jar contains external libraries is too huge to  b= e submitted to the MP by the client.
   So It comes to t= he conclusion that the jar without the external jars could be suited the sit= uation ,however the runnable jar brings the errors.
   How to solve the errors. One way is to set the Hadoop_clas= sspath explicitly when you run the runnable jars.
   Any= other suggestions?

Regards



=E5=8F=91=E8=87=AA=E6=88=91=E7=9A=84 iPhone

=E5=9C=A8 2013-4-4=EF=BC=8C6:47=EF=BC=8CMohammad Tariq <dontariq@gmail.com> =E5= =86=99=E9=81=93=EF=BC=9A

As expected :)
Actually this is something related to Eclipse. When it comes to the creation= of executable jars containing external jars we need to be a bit careful. On= e thing which you could probably try is to select the "Package required libr= aries into generated jar" option while exporting your project as an executab= le jar.

HTH



On Thu, Apr 4, 2013 at 4:10 AM, Varsha Ra= veendran <varsha.raveendran@gmail.com> wrote:
Thank you! You are right!
 I created the jar file u= sing the command line and not eclipse and it worked!




On Thu, Ap= r 4, 2013 at 4:06 AM, Mohammad Tariq <dontariq@gmail.com> wro= te:
It seems to be a non-hadoop i= ssue to me. Is the jar which you are finally creating a runnable jar?We can'= t reference external classes(jars) in a runnable jar file. Everything must b= e contained inside the jar itself.
On Thu, Apr 4, 2013 at 3:= 28 AM, Varsha Raveendran <varsha.raveendran@gmail.com> wrote:
Thanks for replying!

vers= ion : hadoop 1.1.1

I am creating a jar file called ga_test.jar a= nd then placing  it in the build path of another program.
Als= o, I am using eclipse to create another jar file including ga_test.jar as a r= eference library. I do not know why hadoop is taking the filename as rsrc:ga= _test.jar.  (Maybe thats where the ":" is coming into the picture!)
=
Thanks!




On Thu, Apr 4, 2013 at 3:18 AM, M= ohammad Tariq <dontariq@gmail.com> wrote:
Which version are you using? Could you plz s= how me your code, if possible for you. 
Also, how are you running the job? For a detailed explanation you might find= this useful.
On Thu, Apr 4, 2013 at 3:= 09 AM, Varsha Raveendran <varsha.raveendran@gmail.com> wrote:
 But I do not have a ":" in any of my file names.=
What could the other reasons be? I am not able to debug the error= ..

Thank you for replying.


On Thu, Apr 4, 2013 at 2:02 AM, Mohammad T= ariq <dontariq@gmail.com> wrote:
Hello ma'am,

   Please make su= re that you don't have ":" in your files.



On Thu, Apr 4, 2013 at 1:50 AM, Varsha Ra= veendran <varsha.raveendran@gmail.com> wrote:
Hello,

I am facing this error while t= rying to run a jar file on hadoop :

13/04/04 01:48:01 INFO mapred.Job= Client: Cleaning up the staging area hdfs://MT2012158:54310/app/hadoop/tmp/m= apred/staging/hduser/.staging/job_201304032344_0008
Exception in thread "main" java.lang.reflect.InvocationTargetException
&n= bsp;   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Meth= od)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Nat= iveMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delega= tingMethodAccessorImpl.java:43)
    at java.lang.reflect.M= ethod.invoke(Method.java:616)
    at org.eclipse.jdt.inter= nal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Me= thod)
    at sun.reflect.NativeMethodAccessorImpl.invoke(N= ativeMethodAccessorImpl.java:57)
    at sun.reflect.Delega= tingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
&n= bsp;   at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
C= aused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: R= elative path in absolute URI: rsrc:ga_test.jar
    at org.apache.hadoop.fs.Path.initialize(Path.java:148)    at org.apache.hadoop.fs.Path.<init>(Path.java:126)=
    at org.apache.hadoop.mapred.JobClient.copyAndConfigur= eFiles(JobClient.java:841)
    at org.apache.hadoop.mapred= .JobClient.copyAndConfigureFiles(JobClient.java:717)
    at org.apache.hadoop.mapred.JobClient.access$400(JobClien= t.java:179)
    at org.apache.hadoop.mapred.JobClient$2.ru= n(JobClient.java:927)
    at org.apache.hadoop.mapred.JobC= lient$2.run(JobClient.java:912)
    at java.security.Acces= sController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
=     at org.apache.hadoop.security.UserGroupInformation.doAs(U= serGroupInformation.java:1136)
    at org.apache.hadoop.ma= pred.JobClient.submitJobInternal(JobClient.java:912)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient= .java:886)
    at org.apache.hadoop.mapred.JobClient.runJo= b(JobClient.java:1323)
    at edu.iiitb.config.GAMain.laun= ch(Unknown Source)
    at edu.iiitb.config.GAMain.run(Unkn= own Source)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:= 65)
    at edu.iiitb.config.GAMain.runGA(Unknown Source)    at TC1.main(TC1.java:21)
    ... 10 m= ore
Caused by: java.net.URISyntaxException: Relative path in absolute URI= : rsrc:ga_test.jar
    at java.net.URI.checkPath(URI.java:1804)
  &= nbsp; at java.net.URI.<init>(URI.java:752)
    at or= g.apache.hadoop.fs.Path.initialize(Path.java:145)
    ... 2= 7 more

Please help!!!

--
<= b>-Varsha 




--
-Varsha 




--
-Varsha 




--
-Varsha 

= --Apple-Mail-9615C245-4CC1-4AC5-8195-273F3578680F--