hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1100) User's task-logs filling up local disks on the TaskTrackers
Date Wed, 14 Oct 2009 09:34:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765484#action_12765484
] 

Vinod K V commented on MAPREDUCE-1100:
--------------------------------------

This problem is similar to the issues with excessive memory usage by tasks - HADOOP-3581 -
in many ways. And the typical disaster scenario is task logs of particular job *all* wreck
havoc and at once, a bunch of nodes become unusable in the cluster.

A simple short roster of requirements I can think of:
 (1) Excessive logging by one user's tasks should not affect/fail other user's tasks or other
processes running on the same node.
 (2) The user whose task is writing massive logs should be warned/informed about the damage
he /she is doing. This will help in avoiding the same issue to happen again.
 (3) Overall usage of logs by tasks over time on a single node shouldn't affect/fail new tasks
arriving on this node.
 (4) Clean up of a particular task's user logs should be deterministic - users should have
an idea of how long they can see their logs.

> User's task-logs filling up local disks on the TaskTrackers
> -----------------------------------------------------------
>
>                 Key: MAPREDUCE-1100
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1100
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>
> Some user's jobs are filling up TT disks by outrageous logging. mapreduce.task.userlog.limit.kb
is not enabled on the cluster. Disks are getting filled up before task-log cleanup via mapred.task.userlog.retain.hours
can kick in.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message