Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 85397F58D for ; Wed, 17 Apr 2013 07:18:41 +0000 (UTC) Received: (qmail 32324 invoked by uid 500); 17 Apr 2013 07:18:36 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 32066 invoked by uid 500); 17 Apr 2013 07:18:35 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 32040 invoked by uid 99); 17 Apr 2013 07:18:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Apr 2013 07:18:34 +0000 X-ASF-Spam-Status: No, hits=3.2 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lhztop@hotmail.com designates 65.55.111.95 as permitted sender) Received: from [65.55.111.95] (HELO blu0-omc2-s20.blu0.hotmail.com) (65.55.111.95) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Apr 2013 07:18:29 +0000 Received: from BLU0-SMTP466 ([65.55.111.72]) by blu0-omc2-s20.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 17 Apr 2013 00:18:09 -0700 X-EIP: [KvXDfYd/9s85uqV4FjUWHxa0bwXWFjxQ] X-Originating-Email: [lhztop@hotmail.com] Message-ID: Received: from [192.168.45.177] ([122.11.37.122]) by BLU0-SMTP466.phx.gbl over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Wed, 17 Apr 2013 00:18:06 -0700 Date: Wed, 17 Apr 2013 15:17:53 +0800 From: =?UTF-8?B?5p2O5rSq5b+g?= User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: user@hadoop.apache.org CC: Henry Hung Subject: Re: Adding new name node location References: <53DC189E5FAEFA43BFA1BC02431031DB5FD5ABF6B8@WECTMLBOX.winbond.com.tw> <53DC189E5FAEFA43BFA1BC02431031DB5FD5ABF80F@WECTMLBOX.winbond.com.tw> In-Reply-To: <53DC189E5FAEFA43BFA1BC02431031DB5FD5ABF80F@WECTMLBOX.winbond.com.tw> Content-Type: multipart/alternative; boundary="------------060701060302090204070808" X-OriginalArrivalTime: 17 Apr 2013 07:18:07.0196 (UTC) FILETIME=[B55949C0:01CE3B3B] X-Virus-Checked: Checked by ClamAV on apache.org --------------060701060302090204070808 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: quoted-printable modify conf file and restart name node is the best way. you needn't=20 restart the cluster DFS. the files in /backup and /home are the same. =E4=BA=8E 2013/4/17 14:38=2C Henry Hung =E5=86=99=E9=81=93: > > Hi Varun Kumar=2C > > Could you be more elaborate about how the new changes being made to=20 > new name node? > > The scenario in my mind is: > > Suppose old name node metadata contains 100 hdfs files. > > Then I restart by using stop-dfs=2C change config and start-dfs. > > Hadoop will automatically create new name node directory in /backup. > > Now I have one old name node that have metadata of 100 hdfs files and=20 > new name node that have none metadata. > > I put a new file into hdfs=2C the old name node will have 101 and new=20 > name node will have 1. > > Best regards=2C > > Henry > > *From:*varun kumar [mailto:varun.uid@gmail.com] > *Sent:* Wednesday=2C April 17=2C 2013 2:18 PM > *To:* user > *Cc:* MA11 YTHung1 > *Subject:* Re: Adding new name node location > > Hi Henry=2C > > As per your mail Point number 1 is correct. > > After doing these changes metadata will be written in the new partition. > > Regards=2C > > Varun Kumar.P > > On Wed=2C Apr 17=2C 2013 at 11:32 AM=2C Henry Hung > wrote: > > Hi Everyone=2C > > I=E2=80=99m using Hadoop 1.0.4 and only define 1 location for name node f= iles=2C=20 > like this: > > > > dfs.name.dir > > /home/hadoop/hadoop-data/namenode > > > > Now I want to protect my name node files by changing the configuration to= : > > > > dfs.name.dir > > /home/hadoop/hadoop-data/namenode=2C/backup/hadoop/hadoop-data/nam= enode > > > > Where /backup is another mount point. This /backup can be another disk=20 > or from another NFS server. > > My question are: > > 1.Is my procedure correct: do stop-dfs.sh then modify conf=2C and last=20 > start-dfs.sh? > > 2.If answer to no 1 is no=2C then could you provide the correct procedure= ? > > 3.Would the new name node files will auto copy the original name node=20 > files? > > Best regards=2C > > Henry > > ------------------------------------------------------------------------ > > The privileged confidential information contained in this email is=20 > intended for use only by the addressees as indicated by the original=20 > sender of this email. If you are not the addressee indicated in this=20 > email or are not responsible for delivery of the email to such a=20 > person=2C please kindly reply to the sender indicating this fact and=20 > delete all copies of it from your computer and network server=20 > immediately. Your cooperation is highly appreciated. It is advised=20 > that any unauthorized use of confidential information of Winbond is=20 > strictly prohibited=3B and any information in this email irrelevant to=20 > the official business of Winbond shall be deemed as neither given nor=20 > endorsed by Winbond. > > > > --=20 > > Regards=2C > > Varun Kumar.P > > > ------------------------------------------------------------------------ > The privileged confidential information contained in this email is=20 > intended for use only by the addressees as indicated by the original=20 > sender of this email. If you are not the addressee indicated in this=20 > email or are not responsible for delivery of the email to such a=20 > person=2C please kindly reply to the sender indicating this fact and=20 > delete all copies of it from your computer and network server=20 > immediately. Your cooperation is highly appreciated. It is advised=20 > that any unauthorized use of confidential information of Winbond is=20 > strictly prohibited=3B and any information in this email irrelevant to=20 > the official business of Winbond shall be deemed as neither given nor=20 > endorsed by Winbond. --------------060701060302090204070808 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
modify conf file and restart name node is the best way. you needn't restart the cluster DFS.
the files in /backup and /home are the same.
=E4=BA=8E 2013/4/17 14:38=2C Henry Hung =E5=86=99=E9=81=93:

Hi Varun Kumar=2C

=C2=A0

Could =C2=A0you be more elaborate about how the new changes being mad= e to new name node?

=C2=A0

The scenario in my mind is:

Suppose old name node metadata contains 100 hdfs files.

Then I restart by using stop-dfs=2C change config and start-dfs.

Hadoop will automatically create new name node directory in /backup.

Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.

I put a new file into hdfs=2C the old name node will have 101 and new name node will have 1.

=C2=A0

Best regards=2C

Henry

=C2=A0

From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday=2C April 17=2C 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location<= /span>

=C2=A0

Hi Henry=2C

=C2=A0

As per your mail Point number 1 is correct.

=C2=A0

After doing these changes metadata will be written in the new partition.

=C2=A0

Regards=2C

Varun Kumar.P

=C2=A0=

On Wed=2C Apr 17=2C 2013 at 11:32 AM=2C = Henry Hung <=3BYTHun= g1@winbond.com>=3B wrote:

Hi Everyone=2C

=C2=A0

I=E2=80=99m using Hadoop 1.0.4 and only define 1 location for name node files=2C like this:

=C2=A0 <=3Bproperty>=3B

=C2=A0=C2=A0=C2=A0 <=3Bname>=3Bdfs.name.dir<=3B/name>=3B<= /p>

=C2=A0=C2=A0=C2=A0 <=3Bvalue>=3B/home/hadoop/hadoop-data/namenode<=3B/value>=3B

=C2=A0 <=3B/property>=3B

=C2=A0

Now I want to protect my name node files by changing the configuration to:

=C2=A0 <=3Bproperty>=3B

=C2=A0=C2=A0=C2=A0 <=3Bname>=3Bdfs.name.dir<=3B/name>=3B<= /p>

=C2=A0=C2=A0=C2=A0 <=3Bvalue>=3B/home/hadoop/hadoop-data/namenode=2C/backup/hadoop/hadoop-= data/namenode<=3B/value>=3B

=C2=A0 <=3B/property>=3B

=C2=A0

Where /backup is another mount point. This /backup can be another disk or from another NFS server.

=C2=A0

My question are:

1.=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Is my procedure correct: do stop-dfs.sh then modify conf=2C and last start-dfs.sh?

2.=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 If answer to no 1 is no=2C then could you provide the correct procedure?

3.=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Would the new name node files will auto copy the original name node files?

=C2=A0

Best regards=2C

Henry

=C2=A0

The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person=2C please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited=3B and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.



=C2=A0

--

Regards=2C

Varun Kumar.P


The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person=2C please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited=3B and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.

--------------060701060302090204070808--