hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: formatting hdfs without user interaction
Date Fri, 23 Sep 2011 03:48:30 GMT

Writing your own program was overkill.

The 'yes' coreutil is pretty silly, but nifty at the same time. It
accepts an argument, which it would repeat infinitely.


$ yes Y | hadoop namenode -format

Would do it for you.

(Note that in the future release, saner answers will be acceptable,
i.e. y instead of strictly Y, etc.)

Also, two other things:

- What do you mean by 'Yeah I have a secondary namenode as well so 2
directories.'? A secondary namenode uses different directories than
- The prompt only appears when it detects a 'reformat' being happening
- which is very dangerous to do non-interactively. If you do the
-format the first time, on clean dfs.name.dir setups, you will never
receive a prompt.

On Fri, Sep 23, 2011 at 3:53 AM,  <Ivan.Novick@emc.com> wrote:
> Hi Adam,
> Well the "yes" program prints lower case y's and apparently only captial Y
> is accepted.
> But by creating my out "Yes" program that spews Y's to stdout it works :)
> Cheers,
> Ivan
> On 9/22/11 3:02 PM, "Adam Shook" <ashook@clearedgeit.com> wrote:
>>yes | hadoop namenode -format
>>The "yes" program simply outputs 'y' a bunch of times.  "echo yes" will
>>just print "yes" to stdout.
>>-- Adam
>>-----Original Message-----
>>From: Ivan.Novick@emc.com [mailto:Ivan.Novick@emc.com]
>>Sent: Thursday, September 22, 2011 6:00 PM
>>To: common-user@hadoop.apache.org; rajvish@yahoo.com
>>Subject: Re: formatting hdfs without user interaction
>>Hi Raj and Arpit,
>>I think the fact that there are 2 confirmations is causing it not to work:
>>echo "Y" | hadoop namenode -format
>>11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
>>STARTUP_MSG: Starting NameNode
>>STARTUP_MSG:   host = hdm1/
>>STARTUP_MSG:   args = [-format]
>>STARTUP_MSG:   version =
>>STARTUP_MSG:   build = git://hrt8n35.cc1.ygridcore.net/ on branch
>>branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
>>compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
>>Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
>>filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
>>11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
>>SHUTDOWN_MSG: Shutting down NameNode at hdm1/
>> par new generation   total 2831168K, used 150996K [0x00002aaaae210000,
>>0x00002aab6e210000, 0x00002aab6e210000)
>>  eden space 2516608K,   6% used [0x00002aaaae210000, 0x00002aaab75852d0,
>>  from space 314560K,   0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
>>  to   space 314560K,   0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
>> concurrent mark-sweep generation total 30408704K, used 0K
>>[0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
>> concurrent-mark-sweep perm gen total 21248K, used 7333K
>>[0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>>On 9/22/11 2:52 PM, "Raj V" <rajvish@yahoo.com> wrote:
>>>"echo 'Y' | hadoop namenode -format"
>>>should work.,
>>>>From: "Ivan.Novick@emc.com" <Ivan.Novick@emc.com>
>>>>To: common-user@hadoop.apache.org
>>>>Sent: Thursday, September 22, 2011 2:43 PM
>>>>Subject: formatting hdfs without user interaction
>>>>I am trying to automate formatting an HDFS volume.  Is there any way to
>>>>do this without the interaction (and using expect)?
>>No virus found in this message.
>>Checked by AVG - www.avg.com
>>Version: 10.0.1410 / Virus Database: 1520/3912 - Release Date: 09/22/11

Harsh J

View raw message