hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5099) DataNode should warn about unknown files in storage
Date Mon, 11 May 2009 10:47:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707965#action_12707965
] 

Steve Loughran commented on HADOOP-5099:
----------------------------------------

-there could be a lot of unknown files -may be best to list a warning on the count, and give
the first two or three.
-you may want to have the option to completely halt the DN if there are unknowns, this could
be handy to ensure the node hasnt been brought up pointing at the wrong part of the filesystem

> DataNode should warn about unknown files in storage
> ---------------------------------------------------
>
>                 Key: HADOOP-5099
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5099
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Raghu Angadi
>            Assignee: Jakob Homan
>
> DataNode currently just ignores the files it does not know about. There could be a lot
of files left in DataNode's storage that never get noticed or deleted. These files could be
left because of bugs or by a misconfiguration. E.g. while upgrading from 0.17, DN left a lot
of metada files that were not named in correct format for 0.18 (HADOOP-4663).
> The proposal here is simply to make DN print a warning for each of the unknown files
at the start up. This at least gives a way to list all the unknown files and  (equally importantly)
forces a notion of "known" and "unknown" files in the storage.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message