hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2793) [MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs
Date Thu, 23 Feb 2012 23:33:50 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215202#comment-13215202
] 

Bikas Saha commented on MAPREDUCE-2793:
---------------------------------------

The hashCode difference was because the ApplicationId internal to JobId was different. The
test creates 3 jobs with the same app id. However currently, having jobid == appid is baked
into a lot of code including the one used to fix the inconsistency in names. The test would
create a list of 3 jobs with id's 0,1,2 and app id =0. The it would fetch the all the jobs
from the webserver, pick the first job and verify that it exists in its list. Hence when the
new code in the webserver was used to generate the jobid from the jobid string, it returned
a job id with app id equal to the job id. This job id would have a different app id than the
one in the test list except for when the job id was 0. So when the first job in the list was
job id 0 then the test would pass, and otherwise it would fail. The order in the list would
change with each run because the list was a hash map.
                
> [MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs 
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2793
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2793
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: Bikas Saha
>            Priority: Critical
>             Fix For: 0.23.2
>
>         Attachments: MAPREDUCE-2793-branch-0.23.patch, MAPREDUCE-2793-branch-0.23.patch,
MAPREDUCE-2793-branch-0.23.patch, MAPREDUCE-2793-branch-0.23.patch, MAPREDUCE-2793-branch-0.23.patch,
MAPREDUCE-2793-branch-0.23.patch, MAPREDUCE-2793.patch
>
>
> appIDs, jobIDs and attempt/container ids are not consistently named in the logs, console
and UI. For consistency purpose, they all have to follow a common naming convention.
> Currently, 
> For appID
> =========
> On the RM UI: app_1308259676864_5 
> On the JHS UI: No appID 
> Console/logs: No appID
> mapred-local dirs are named as: application_1308259676864_0005
> For jobID
> =========
> On the RM UI: job_1308259676864_5_5 
> JHS UI: job_1308259676864_5_5 
> Console/logs: job_1308259676864_0005
> mapred-local dirs are named as: No jobID
> For attemptID
> ============
> On the RM UI: attempt_1308259676864_5_5_m_24_0
> JHS attempt_1308259676864_5_5_m_24_0
> Console/logs: attempt_1308259676864_0005_m_000024_0
> mapred-local dirs are named as: container_1308259676864_0005_000024

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message