hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2716) MR279: MRReliabilityTest job fails because of missing job-file.
Date Wed, 03 Aug 2011 14:12:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078759#comment-13078759
] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-2716:
----------------------------------------------------

Apologies for the delayed reply, I was tied up with one of those bigger patches.

bq. .. that staging area directory seems to include the user name and ".staging" as well as
the value from the "yarn.apps.stagingDir"
Good catch, that's correct! In fact, given this, {{ClientServiceDelegate.getJobStatus()}}
using the key from configuration directly is actually a bug.

For fixing this, we should be obtaining the job-file by simply using the same API from MRApps
in both getJobStatus() and getAllJobs(). Or may be just add another API {{MRApps.getJobFile(Configuration
conf, String user, JobID jobID)}}


> MR279: MRReliabilityTest job fails because of missing job-file.
> ---------------------------------------------------------------
>
>                 Key: MAPREDUCE-2716
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2716
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Jeffrey Naisbitt
>            Assignee: Jeffrey Naisbitt
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2716-v2.patch, MAPREDUCE-2716-v3.patch, MAPREDUCE-2716.patch
>
>
> The ApplicationReport should have the jobFile (e.g. hdfs://localhost:9000/tmp/hadoop-<USER>/mapred/staging/<USER>/.staging/job_201107121640_0001/job.xml)
> Without it, jobs such as MRReliabilityTest fail with the following error (caused by the
fact that jobFile is hardcoded to "" in TypeConverter.java):
> e.g. java.lang.IllegalArgumentException: Can not create a Path from an empty string
>         at org.apache.hadoop.fs.Path.checkPathArg(Path.java:88)
>         at org.apache.hadoop.fs.Path.<init>(Path.java:96)
>         at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:445)
>         at org.apache.hadoop.mapreduce.Cluster.getJobs(Cluster.java:104)
>         at org.apache.hadoop.mapreduce.Cluster.getAllJobs(Cluster.java:218)
>         at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:757)
>         at org.apache.hadoop.mapred.JobClient.jobsToComplete(JobClient.java:741)
>         at org.apache.hadoop.mapred.ReliabilityTest.runTest(ReliabilityTest.java:219)
>         at org.apache.hadoop.mapred.ReliabilityTest.runSleepJobTest(ReliabilityTest.java:133)
>         at org.apache.hadoop.mapred.ReliabilityTest.run(ReliabilityTest.java:116)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
>         at org.apache.hadoop.mapred.ReliabilityTest.main(ReliabilityTest.java:504)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>         at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
>         at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:111)
>         at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:118)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:192)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message