hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karam Singh (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3304) [HOD] logcondense fails if DFS has files that are not log files, but match a certain pattern
Date Tue, 13 May 2008 09:22:55 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Karam Singh updated HADOOP-3304:
--------------------------------


When dynamicdfs is used and logcondense fails to delete tasktracker logs if -n options e.g
-:
[
 python logcondense.py -p ~/hadoop-0.17.0/bin/hadoop -d 7 -c ~/hadoop-conf -l /user 
]

logcondense tries to delete 0-tasketracker e.g -:
[
~/hadoop-0.17.0/bin/hadoop --config  ~/hadoop-conf dfs -rm  /user/user-id/hod-logs/jobid.hostname/0-tasktracker-*
]
where  0-tasktracker-* file are not there because in case of dynamic dfs logs are like 0-datanode-*
and 1-tasktracker-*


> [HOD] logcondense fails if DFS has files that are not log files, but match a certain
pattern
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3304
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3304
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/hod
>    Affects Versions: 0.16.2
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>            Priority: Blocker
>             Fix For: 0.16.4
>
>         Attachments: 3304.1.patch, 3304.2.patch, 3304.patch
>
>
> logcondense works by listing files in dfs and match them against a certain pattern. This
pattern is incorrect in the sense that it can potentially match files that are not log files.
And this can cause it to fail and not delete files correctly as it should.
> It should use a correct regular expression that will only list log files. Also, possibly
it should log a stack trace if it happens, but continue to delete files and not stop.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message