hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yypvsxf19870706 <yypvsxf19870...@gmail.com>
Subject Re: Error while running MapR program on multinode configuration
Date Thu, 04 Apr 2013 04:41:57 GMT
Hi Mohammad

   Thanks for your response .

    As far as I know,the jars which the client submit to the MP through the net in DistributedCache
mode because of the jar localization .
   The point that I aim at is we need to reduce the size of the objects we submit through
network ,which will save the bandwidth and the time to run the job.
   Maybe I was wrong ,discussion will be appreciated.

发自我的 iPhone

在 2013-4-4,7:47,Mohammad Tariq <dontariq@gmail.com> 写道:

> DistributedCache??
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Thu, Apr 4, 2013 at 5:01 AM, yypvsxf19870706 <yypvsxf19870706@gmail.com> wrote:
>> Hi all
>> 
>>    However,I do think the export jar contains external libraries is too huge to 
be submitted to the MP by the client.
>>    So It comes to the conclusion that the jar without the external jars could be
suited the situation ,however the runnable jar brings the errors.
>>    How to solve the errors. One way is to set the Hadoop_classspath explicitly when
you run the runnable jars.
>>    Any other suggestions?
>> 
>> Regards
>> 
>> 
>> 
>> 发自我的 iPhone
>> 
>> 在 2013-4-4,6:47,Mohammad Tariq <dontariq@gmail.com> 写道:
>> 
>>> As expected :)
>>> Actually this is something related to Eclipse. When it comes to the creation
of executable jars containing external jars we need to be a bit careful. One thing which you
could probably try is to select the "Package required libraries into generated jar" option
while exporting your project as an executable jar.
>>> 
>>> HTH
>>> 
>>> Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>> cloudfront.blogspot.com
>>> 
>>> 
>>> On Thu, Apr 4, 2013 at 4:10 AM, Varsha Raveendran <varsha.raveendran@gmail.com>
wrote:
>>>> Thank you! You are right!
>>>>  I created the jar file using the command line and not eclipse and it worked!
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Thu, Apr 4, 2013 at 4:06 AM, Mohammad Tariq <dontariq@gmail.com>
wrote:
>>>>> It seems to be a non-hadoop issue to me. Is the jar which you are finally
creating a runnable jar?We can't reference external classes(jars) in a runnable jar file.
Everything must be contained inside the jar itself.
>>>>> 
>>>>> Warm Regards,
>>>>> Tariq
>>>>> https://mtariq.jux.com/
>>>>> cloudfront.blogspot.com
>>>>> 
>>>>> 
>>>>> On Thu, Apr 4, 2013 at 3:28 AM, Varsha Raveendran <varsha.raveendran@gmail.com>
wrote:
>>>>>> Thanks for replying! 
>>>>>> 
>>>>>> version : hadoop 1.1.1
>>>>>> 
>>>>>> I am creating a jar file called ga_test.jar and then placing  it
in the build path of another program. 
>>>>>> Also, I am using eclipse to create another jar file including ga_test.jar
as a reference library. I do not know why hadoop is taking the filename as rsrc:ga_test.jar.
 (Maybe thats where the ":" is coming into the picture!)
>>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Thu, Apr 4, 2013 at 3:18 AM, Mohammad Tariq <dontariq@gmail.com>
wrote:
>>>>>>> Which version are you using? Could you plz show me your code,
if possible for you. 
>>>>>>> Also, how are you running the job? For a detailed explanation
you might find this useful.
>>>>>>> 
>>>>>>> Warm Regards,
>>>>>>> Tariq
>>>>>>> https://mtariq.jux.com/
>>>>>>> cloudfront.blogspot.com
>>>>>>> 
>>>>>>> 
>>>>>>> On Thu, Apr 4, 2013 at 3:09 AM, Varsha Raveendran <varsha.raveendran@gmail.com>
wrote:
>>>>>>>>  But I do not have a ":" in any of my file names. 
>>>>>>>> What could the other reasons be? I am not able to debug the
error..
>>>>>>>> 
>>>>>>>> Thank you for replying.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Thu, Apr 4, 2013 at 2:02 AM, Mohammad Tariq <dontariq@gmail.com>
wrote:
>>>>>>>>> Hello ma'am,
>>>>>>>>> 
>>>>>>>>>    Please make sure that you don't have ":" in your files.
>>>>>>>>> 
>>>>>>>>> Warm Regards,
>>>>>>>>> Tariq
>>>>>>>>> https://mtariq.jux.com/
>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Thu, Apr 4, 2013 at 1:50 AM, Varsha Raveendran <varsha.raveendran@gmail.com>
wrote:
>>>>>>>>>> Hello,
>>>>>>>>>> 
>>>>>>>>>> I am facing this error while trying to run a jar
file on hadoop :
>>>>>>>>>> 
>>>>>>>>>> 13/04/04 01:48:01 INFO mapred.JobClient: Cleaning
up the staging area hdfs://MT2012158:54310/app/hadoop/tmp/mapred/staging/hduser/.staging/job_201304032344_0008
>>>>>>>>>> Exception in thread "main" java.lang.reflect.InvocationTargetException
>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>>>>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>>>     at java.lang.reflect.Method.invoke(Method.java:616)
>>>>>>>>>>     at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>>>>>>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>>>>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>>>     at java.lang.reflect.Method.invoke(Method.java:616)
>>>>>>>>>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
Relative path in absolute URI: rsrc:ga_test.jar
>>>>>>>>>>     at org.apache.hadoop.fs.Path.initialize(Path.java:148)
>>>>>>>>>>     at org.apache.hadoop.fs.Path.<init>(Path.java:126)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:841)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:717)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:179)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:927)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
>>>>>>>>>>     at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>>     at javax.security.auth.Subject.doAs(Subject.java:416)
>>>>>>>>>>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
>>>>>>>>>>     at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323)
>>>>>>>>>>     at edu.iiitb.config.GAMain.launch(Unknown Source)
>>>>>>>>>>     at edu.iiitb.config.GAMain.run(Unknown Source)
>>>>>>>>>>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>>>>>     at edu.iiitb.config.GAMain.runGA(Unknown Source)
>>>>>>>>>>     at TC1.main(TC1.java:21)
>>>>>>>>>>     ... 10 more
>>>>>>>>>> Caused by: java.net.URISyntaxException: Relative
path in absolute URI: rsrc:ga_test.jar
>>>>>>>>>>     at java.net.URI.checkPath(URI.java:1804)
>>>>>>>>>>     at java.net.URI.<init>(URI.java:752)
>>>>>>>>>>     at org.apache.hadoop.fs.Path.initialize(Path.java:145)
>>>>>>>>>>     ... 27 more
>>>>>>>>>> 
>>>>>>>>>> Please help!!!
>>>>>>>>>> 
>>>>>>>>>> -- 
>>>>>>>>>> -Varsha 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> -- 
>>>>>>>> -Varsha 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -- 
>>>>>> -Varsha 
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> -Varsha 
> 

Mime
View raw message