hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Ryan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-718) configuration parameter to prevent accidental formatting of HDFS filesystem
Date Wed, 04 Nov 2009 00:27:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773303#action_12773303
] 

Andrew Ryan commented on HDFS-718:
----------------------------------

The issue we're trying to address is one where the user thinks they are formatting the correct
namenode, but actually are formatting the wrong namenode. In our environment we have a team
of people administering multiple clusters, some of which may be important production ones
which should never be formatted, and some of which are development or test clusters which
might get formatted more frequently. And the admins may have multiple windows open on their
screen at the same time showing different clusters, which can increase the chances of errors.

This patch doesn't protect against 'rm -rf', the administrator changing the configuration,
or other sorts of malice, but that isn't the intention of the patch. Nor is this patch meant
to be a substitute for regular backups of the filesystem image. The intention is to provide
an additional safeguard against administrative error. Backups do not prevent operator error
; they merely allow recovery from it. Better to never lose your filesystem image than to have
to try and recover it from a backup.

> configuration parameter to prevent accidental formatting of HDFS filesystem
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-718
>                 URL: https://issues.apache.org/jira/browse/HDFS-718
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.22.0
>         Environment: Any
>            Reporter: Andrew Ryan
>            Assignee: Andrew Ryan
>            Priority: Minor
>         Attachments: HDFS-718.patch.txt
>
>
> Currently, any time the NameNode is not running, an HDFS filesystem will accept the 'format'
command, and will duly format itself. There are those of us who have multi-PB HDFS filesystems
who are really quite uncomfortable with this behavior. There is "Y/N" confirmation in the
format command, but if the formatter genuinely believes themselves to be doing the right thing,
the filesystem will be formatted.
> This patch adds a configuration parameter to the namenode, dfs.namenode.support.allowformat,
which defaults to "true," the current behavior: always allow formatting if the NameNode is
down or some other process is not holding the namenode lock. But if dfs.namenode.support.allowformat
is set to "false," the NameNode will not allow itself to be formatted until this config parameter
is changed to "true".
> The general idea is that for production HDFS filesystems, the user would format the HDFS
once, then set dfs.namenode.support.allowformat to "false" for all time.
> The attached patch was generated against trunk and +1's on my test machine. We have a
0.20 version that we are using in our cluster as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message