Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <1044843187.26571267597287372.JavaMail.jira@brutus.apache.org>
Date: Wed, 3 Mar 2010 06:21:27 +0000 (UTC)
From: "Amareshwari Sriramadasu (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Updated: (MAPREDUCE-927) Cleanup of task-logs should happen
 in TaskTracker instead of the Child
In-Reply-To: <353529923.1251455879295.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sriramadasu updated MAPREDUCE-927:
----------------------------------------------

    Attachment: patch-927-2.txt

Thanks Ravi for the explanation. I have undone the change.

bq. Shouldn't clearOldUserLogs() be done as part of constructor itself?
No. Again for the same reason that the thread is a daemon thread and is constructed only once.

bq.  Rename the class to TaskLogsMonitor, so that we are consistent going forward with MAPREDUCE-1100.
Renamed it to UserLogCleaner. 

bq. We should create some attempt dirs also in the joblog dir with appropriate permissions and verify the proper cleanup.
All this code path is tested with proper permissions in TestTaskTrackerLocalization and TestLocalizationWIthLinuxTaskController.

bq. Should TaskLogs section be changed to explicitly specify the new directory hierarchy?
Did not do this. I don't think we want to publish the directory hierarchy for users in this jira.

Patch incorporates almost all the review comments.


> Cleanup of task-logs should happen in TaskTracker instead of the Child
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-927
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-927
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: patch-927-1.txt, patch-927-2.txt, patch-927.txt
>
>
> Task logs' cleanup is being done in Child now. This is undesirable atleast for two reasons: 1) failures while cleaning up will affect the user's tasks, and 2) the task's wall time will get affected due to operations that TT actually should own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.