hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3342) job history doesn't show job queue
Date Mon, 07 Nov 2011 22:00:54 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145862#comment-13145862
] 

jiraposter@reviews.apache.org commented on MAPREDUCE-3342:
----------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2756/#review3088
-----------------------------------------------------------



hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
<https://reviews.apache.org/r/2756/#comment6893>

    Could you update the comment a bit.  It kind of loses the intention to not fail if the
format is not completely correct. Or just add in the try/catch blocks for the number format
exception on each parse int/parselong


- Robert


On 2011-11-07 21:54:28, Jonathan Eagles wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2756/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-11-07 21:54:28)
bq.  
bq.  
bq.  Review request for Tom Graves, Robert Evans, Ravi Prakash, Mark Holderbaugh, and Anupam
Seth.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  The job history server doesn't show the queue the jobwas run in. It is inserted into
the job history file.
bq.  
bq.  It seems like this should be part of the Job interface.
bq.  
bq.  JobImpl current gets it from the job config to insert into the history.
bq.  
bq.  ------------------
bq.  Some more background. jobhistory/job jobs display a single CompletedJob instance which
is populated by parsing the jobhistory file contents.
bq.  jobhistory/app jobs display all jobs in the history via PartialJob instances which are
populated by parsing the jobhistory file names only, not by looking at the contents.
bq.  
bq.  By changing the jobhistory file naming scheme, I have disabled rigid parsing of job history
file names and try to get as much information from them as we can.
bq.  
bq.  
bq.  This addresses bug MAPREDUCE-3342.
bq.      http://issues.apache.org/jira/browse/MAPREDUCE-3342
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
07f8ecc 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/Job.java
fdba78d 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
ea8bbdf 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
7a6e1f0 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
5669070 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
f22d51c 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobIndexInfo.java
9f83b70 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java
9de3dcd 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
71680d5 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
fc808e5 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
c6e3b64 
bq.    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java
6ef7bf0 
bq.  
bq.  Diff: https://reviews.apache.org/r/2756/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  In addition to unit tests. I have manually tested coexistence of old and new job history
files, both display the queue name on the jobhistory/job page correctly. On the jobhistory/app
page old job history file queue names will be displayed as null and new history file queue
names will be displayed correctly. 
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jonathan
bq.  
bq.


                
> job history doesn't show job queue
> ----------------------------------
>
>                 Key: MAPREDUCE-3342
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3342
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Thomas Graves
>            Assignee: Jonathan Eagles
>            Priority: Critical
>         Attachments: MAPREDUCE-3342.patch, MAPREDUCE-3342.patch
>
>
> The job history server doesn't show the queue the jobwas run in.  It is inserted into
the job history file.
> It seems like this should be part of the Job interface.
> JobImpl current gets it from the job config to insert into the history.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message