hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-4033) MiniMRClientClusterFactory is not setting the temp dir correctly in the conf used to init MiniMRYarnCluster
Date Fri, 30 Mar 2012 11:54:28 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alejandro Abdelnur resolved MAPREDUCE-4033.
-------------------------------------------

    Resolution: Invalid
      Assignee: Alejandro Abdelnur

Oozie testcases are not using the MiniMR configuration to submit a job, thus missing settings
shared between MiniMR and the AMs.

Still it seems to be an issue when the staging HDFS directory of the AM and the JH differ
and the app never ends. will verify and open JIRA if appropriate.
                
> MiniMRClientClusterFactory is not setting the temp dir correctly in the conf used to
init MiniMRYarnCluster
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4033
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4033
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.3
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: minicluster-oozie-pig.txt
>
>
> Oozie testcases are failing randomly because MR2 reports the job as unknown.
> This seems to happen when Oozie queries via JobClient.getJob(<JOBID>) for a <JOBID>
that just finished.
> {code}
> org.apache.oozie.action.ActionExecutorException: JA017: Unknown hadoop job [job_1332176678205_0011]
associated with action [0000000-120319101023910-oozie-tucu-W@pig-action].  Failing this action!
> {code}
> Oozie reports this error when JobClient.getJob(<JOBID>) returns NULL.
> Looking at the mini cluster logs the job definitely run.
> {code}
>  find . -name "*1332176678205_0011*"
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1332176678205_0011
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1332176678205_0011/container_1332176678205_0011_01_000002
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1332176678205_0011/container_1332176678205_0011_01_000001
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1332176678205_0011
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1332176678205_0011/container_1332176678205_0011_01_000002
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1332176678205_0011/container_1332176678205_0011_01_000001
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1332176678205_0011
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1332176678205_0011/container_1332176678205_0011_01_000002
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1332176678205_0011/container_1332176678205_0011_01_000001
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_1/application_1332176678205_0011
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_1/application_1332176678205_0011/container_1332176678205_0011_01_000002
> ./core/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_1/application_1332176678205_0011/container_1332176678205_0011_01_000001
> {code}
> It seems there is a gap until the the job is avail in the JH server.
> If this gap is unavoidable we need to ensure Oozie always waits at least the gap time
before querying for a job.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message