hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shinichi Yamashita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5392) Fix "mapred job -history all" command
Date Wed, 17 Jul 2013 21:33:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13711651#comment-13711651
] 

Shinichi Yamashita commented on MAPREDUCE-5392:
-----------------------------------------------

I examined why a test in "TestJobHistoryParsing" succeeded. Then, node name "localhost" was
output in JobHistory file for tests.

{quote}
 \{"type":"MAP_ATTEMPT_STARTED","event":\{"org.apache.hadoop.mapreduce.jobhistory.TaskAttemptStarted":\{"taskid":"task_0_0000_m_000000","taskType":"MAP","attemptId":"attempt_0_0000_m_000000_0","startTime":1374095073945,"trackerName":"localhost","httpPort":8042,"shufflePort":-1,"containerId":"container_0_0000_01_000000","locality":"OFF_SWITCH","avataar":"VIRGIN"\}\}\}
{quote}

On the other hand, the URL of the task log was displayed with "t".

{quote}
MAP task list for job_0_0000
TaskId          StartTime       FinishTime      HostName        Error   TaskLogs
====================================================
attempt_0_0000_m_000002_0       17-7-2013 14:23:52      17-7-2013 14:23:52 (0sec)       localhost
      http://t:8042/tasklog?attemptid=attempt_0_0000_m_000002_0
attempt_0_0000_m_000001_0       17-7-2013 14:23:52      17-7-2013 14:23:52 (0sec)       localhost
      http://t:8042/tasklog?attemptid=attempt_0_0000_m_000001_0
attempt_0_0000_m_000000_0       17-7-2013 14:23:52      17-7-2013 14:23:52 (0sec)       localhost
      http://t:8042/tasklog?attemptid=attempt_0_0000_m_000000_0
{quote}

This is because "localhos" was deleted among "localhost" in HostUtil.convertTrackerNameToHostName().
In other words, it may be said that a test cord was insufficient.
I will fix it about TestJobHistoryParsing.java .
                
> Fix "mapred job -history all" command
> -------------------------------------
>
>                 Key: MAPREDUCE-5392
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5392
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 3.0.0, 2.0.5-alpha
>            Reporter: Shinichi Yamashita
>            Assignee: Shinichi Yamashita
>            Priority: Minor
>             Fix For: 3.0.0
>
>         Attachments: MAPREDUCE-5392.patch, MAPREDUCE-5392.patch
>
>
> When I use an "all" option by "mapred job -history" comamnd, the following exceptions
are displayed and do not work.
> {code}
> Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out
of range: -3
>         at java.lang.String.substring(String.java:1875)
>         at org.apache.hadoop.mapreduce.util.HostUtil.convertTrackerNameToHostName(HostUtil.java:49)
>         at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.getTaskLogsUrl(HistoryViewer.java:459)
>         at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.printAllTaskAttempts(HistoryViewer.java:235)
>         at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.print(HistoryViewer.java:117)
>         at org.apache.hadoop.mapreduce.tools.CLI.viewHistory(CLI.java:472)
>         at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:313)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>         at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1233)
> {code}
> This is because a node name recorded in History file is not given "tracker_". Therefore
it makes modifications to be able to read History file even if a node name is not given by
"tracker_".
> In addition, it fixes the URL of displayed task log.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message