hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-927) Cleanup of task-logs should happen in TaskTracker instead of the Child
Date Sun, 19 Dec 2010 03:34:04 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972898#action_12972898

Todd Lipcon commented on MAPREDUCE-927:

With this patch I've seen occasional NPEs in jvmFinished - it seems initalContext.task is
ending up null. Have you guys seen this as well? My current theory is that it's because jvmToRunningTask
is accessed without synchronization in JvmRunner.runChild, so if that HashMap.get() overlaps
with a rehashing due to a concurrent put, it will return a false null. My reasoning is that
when I've seen this happen it's within a couple ms of another task being spawned or removed.

Also, what's the status of this for 22? Should it be considered a blocker?

> Cleanup of task-logs should happen in TaskTracker instead of the Child
> ----------------------------------------------------------------------
>                 Key: MAPREDUCE-927
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-927
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>         Attachments: patch-927-1.txt, patch-927-2.txt, patch-927-3.txt, patch-927-4.txt,
patch-927-5-dist.txt, patch-927.txt
> Task logs' cleanup is being done in Child now. This is undesirable atleast for two reasons:
1) failures while cleaning up will affect the user's tasks, and 2) the task's wall time will
get affected due to operations that TT actually should own.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message