hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peeyush Bishnoi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5022) [HOD] logcondense should delete all hod logs for a user, including jobtracker logs
Date Thu, 15 Jan 2009 11:18:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Peeyush Bishnoi updated HADOOP-5022:

    Attachment: hadoop-5022.txt

Patch is attached for logcondense.py that will optionally delete the JobTracker logs and also
update the  logcondense.py documentation . In fact the patch will delete the complete job
directory inside  hod-logs in DFS if option "-a" or "--all" is set to 'true' . 

TaskTracker logs gets deleted if option "-a" or "--all" is set to 'false' or  if option is
not set . By default option is set to 'false'. 

For example:
python logcondense.py -p ~/hadoop-0.17.0/bin/hadoop -d 7 -c ~/hadoop-conf -l /user -a true


> [HOD] logcondense should delete all hod logs for a user, including jobtracker logs
> ----------------------------------------------------------------------------------
>                 Key: HADOOP-5022
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5022
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/hod
>            Reporter: Hemanth Yamijala
>            Assignee: Peeyush Bishnoi
>            Priority: Blocker
>             Fix For: 0.18.3
>         Attachments: hadoop-5022.txt
> Currently, logcondense.py does not delete jobtracker logs that it uploads to the DFS
when the HOD cluster is deallocated. This will result in the hod-logs directory to slowly
accumulate a whole bunch of jobtracker logs. Particularly for users who run a lot of user
jobs, this could fill up the namespace.  Further these directories will cause the logcondense
program to keep repeatedly looking at these directories stressing out the namenode. So, logcondense.py
should optionally also delete the jobtracker logs.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message