hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure
Date Thu, 04 Jan 2007 22:16:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462336
] 

Owen O'Malley commented on HADOOP-801:
--------------------------------------

1. I don't think it is a good idea to break control constructs between jsp escapes.:

   <% if (flag) { %>
       random text
   <% } %>

seems like bad style. I'd rather see:
<% if (flag) { out.write("random text"); } %>

2. taskCompletionEvents should be initialized to a capacity of maps + reduces + 10 instead
of the default 10.

3. JobInProgress line ~294 has duplicated code for adding the new event depending on whether
the task failed or succeeded. It should just be:
  taskCompletionEvents.add(..., (state == TaskStatus.Status.SUCCEEDED ? 
                                                          TaskCompletionEvent.Status.SUCCEEDED
:
                                                          TaskCompletionEvent.Status.FAILED),
... );
4. both  JobInProgress.getTaskCompletionEvents and JobTracker.getTaskCompletionEvents always
creates a 0 length array of TaskCompletionEvents. It should either be a static final EMPTY_TASK_COMPLETION_EVENTS
or only allocated when there are no events. We could use null to signal no events, but that
is less user friendly.

5. the tasklog.jsp should check to make sure that plaintext is actually a boolean and support
setting false. (Currently, if it is set at all, it assumes it is true.)

6. the url in the TaskCompletionEvent should include the taskid.

7. printHttpFile depends on content length being set, but the jsp doesn't set the content
length.

8. the error message in printHttpFile was copied out of its context from MapOutputLocation.getFile
and talks about map output



> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws
IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the
position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide
the start of the fix for HADOOP-248.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message