hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2415) Distribute TaskTracker userlogs onto multiple disks
Date Fri, 01 Apr 2011 08:51:05 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014474#comment-13014474
] 

Ravi Gummadi commented on MAPREDUCE-2415:
-----------------------------------------

>> to strengthen TaskTracker reliability w.r.t disk failures.

In addition to that, user logs can grow to bigger sizes and can stay alive for more time by
setting mapred.userlog.retain.hours to higher value because of spreading them on to multiple
disks.

> Distribute TaskTracker userlogs onto multiple disks
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-2415
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2415
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task-controller, tasktracker
>    Affects Versions: 0.20.204.0
>            Reporter: Bharath Mundlapudi
>            Assignee: Bharath Mundlapudi
>             Fix For: 0.20.204.0
>
>         Attachments: MAPREDUCE-2415-1.patch, TaskTracker Userlogs Design.pdf
>
>
> Currently, userlogs directory in TaskTracker is placed under hadoop.log.dir like <hadoop.log.dir>/userlogs.
I am proposing to spread these userlogs onto multiple configured mapred.local.dirs to strengthen
TaskTracker reliability w.r.t disk failures.  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message