hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2415) Distribute TaskTracker userlogs onto multiple disks
Date Mon, 04 Apr 2011 18:37:05 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015539#comment-13015539
] 

Owen O'Malley commented on MAPREDUCE-2415:
------------------------------------------

I'd also change the "Invalid local dir count" error message to "No valid local directories."

I'd also add an error message in get_nth_local_dir to say, "Invalid index %d for %d local
directories."

With those two covered, you don't need to include the log for "Invalid local dir."

It looks like the new error messages in initialize_job after calling open_file_as_task_tracker
will produce duplicated messages. Same after change_user.

> Distribute TaskTracker userlogs onto multiple disks
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-2415
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2415
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task-controller, tasktracker
>    Affects Versions: 0.20.204.0
>            Reporter: Bharath Mundlapudi
>            Assignee: Bharath Mundlapudi
>             Fix For: 0.20.204.0
>
>         Attachments: MAPREDUCE-2415-1.patch, TaskTracker Userlogs Design.pdf
>
>
> Currently, userlogs directory in TaskTracker is placed under hadoop.log.dir like <hadoop.log.dir>/userlogs.
I am proposing to spread these userlogs onto multiple configured mapred.local.dirs to strengthen
TaskTracker reliability w.r.t disk failures.  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message