hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-927) Cleanup of task-logs should happen in TaskTracker instead of the Child
Date Wed, 03 Mar 2010 06:21:27 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu updated MAPREDUCE-927:
----------------------------------------------

    Attachment: patch-927-2.txt

Thanks Ravi for the explanation. I have undone the change.

bq. Shouldn't clearOldUserLogs() be done as part of constructor itself?
No. Again for the same reason that the thread is a daemon thread and is constructed only once.

bq.  Rename the class to TaskLogsMonitor, so that we are consistent going forward with MAPREDUCE-1100.
Renamed it to UserLogCleaner. 

bq. We should create some attempt dirs also in the joblog dir with appropriate permissions
and verify the proper cleanup.
All this code path is tested with proper permissions in TestTaskTrackerLocalization and TestLocalizationWIthLinuxTaskController.

bq. Should TaskLogs section be changed to explicitly specify the new directory hierarchy?
Did not do this. I don't think we want to publish the directory hierarchy for users in this
jira.

Patch incorporates almost all the review comments.



> Cleanup of task-logs should happen in TaskTracker instead of the Child
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-927
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-927
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: patch-927-1.txt, patch-927-2.txt, patch-927.txt
>
>
> Task logs' cleanup is being done in Child now. This is undesirable atleast for two reasons:
1) failures while cleaning up will affect the user's tasks, and 2) the task's wall time will
get affected due to operations that TT actually should own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message