hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2165) Augment JobHistory to store tasks' userlogs
Date Fri, 19 Sep 2008 11:18:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod K V updated HADOOP-2165:

    Attachment: HADOOP-2165-20080919.2.txt

New patch.

bq. 1) Hostname is also written to the history and hence that can be used instead of extracting
it from the tracker-name.
HOSTNAME also includes rack-name and i'll need to parse that too to get the tasktracker hostname,
which is difficult than getting it from TRACKER_NAME. Leaving it like that.

But, why do we need both of these in the first place, can't we unify them and have a single
key-val pair? We may also want to knock of "tracker_" prefix if that is not needed, or may
be just provide a api to get the actual hostname stripping off this prefix. Will file a JIRA
if need be, to address these.

bq. 2) Hostnames are always expected and hence we can avoid the check for their existence.
If hostnames are missing then its a bug and should not be masked.
I have empty string checks also for HTTP_PORT. Leaving the checks as they are now.

bq.3) Factor out the common code to do with getting the task-log url. I would prefer not to
handcraft it everywhere. May be in TaskTracker.java?
Introduced public static TaskLogServlet.getTaskLogUrl(String taskTrackerHostName, String httpPort,
String taskAttemptID) for this. Modified taskdetails.jsp and jobfailures.jsp to use the same.

> Augment JobHistory to store tasks' userlogs
> -------------------------------------------
>                 Key: HADOOP-2165
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2165
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Arun C Murthy
>            Assignee: Vinod K V
>             Fix For: 0.19.0
>         Attachments: HADOOP-2165-20080910.1.txt, HADOOP-2165-20080912.txt, HADOOP-2165-20080919.1.txt,
HADOOP-2165-20080919.2.txt, patch_userlog_1.4.3.txt
> It will be very useful to be able to see the job's userlogs (the stdout/stderr/syslog
of the tasks) from the JobHistory page. It will greatly aid in debugging etc.
> At the very minimum we should have links from the JobHistory to the logs on the TT.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message