hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-528) Add dfsadmin -waitDatanodes feature to block until DNs have reported
Date Wed, 05 Aug 2009 00:32:14 GMT

     [ https://issues.apache.org/jira/browse/HDFS-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated HDFS-528:

    Attachment: hdfs-528.txt

This patch implements dfsadmin -waitDatanodes.

For testing, I augmented TestHDFSCLI and also ran through the following on the command line:

$ ./bin/hdfs dfsadmin -help waitDatanodes
-waitDatanodes [timeout] [numNodes]: Wait for datanodes to report.
                Waits for the specified number of Datanodes to report as alive to the
                Namenode. A timeout can be specified in seconds - if the specified
                number of nodes do not report within the timeout, returns a non-zero
                exit code. If the number of nodes is not specified, defaults to the
                value of dfs.replication.min. If the timeout is not specified, defaults
                to 5 minutes.
$ ./bin/hdfs dfsadmin -waitDatanodes
SUCCESS: 1 datanodes have reported.
$ echo $?
$ ./bin/hdfs dfsadmin -waitDatanodes 5 10
Timed out waiting for 10 datanodes after 5 seconds
$ echo $?

One example use case for this tool is in the EC2 scripts - even though the master has started
up, the user needs to wait until at least a few DNs have reported before he or she can start
writing to the cluster.

> Add dfsadmin -waitDatanodes feature to block until DNs have reported
> --------------------------------------------------------------------
>                 Key: HDFS-528
>                 URL: https://issues.apache.org/jira/browse/HDFS-528
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: scripts
>    Affects Versions: 0.21.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-528.txt
> When starting up a fresh cluster programatically, users often want to wait until DFS
is "writable" before continuing in a script. "dfsadmin -safemode wait" doesn't quite work
for this on a completely fresh cluster, since when there are 0 blocks on the system, 100%
of them are accounted for before any DNs have reported.
> This JIRA is to add a command which waits until a certain number of DNs have reported
as alive to the NN.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message