Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 034DAF999 for ; Wed, 17 Apr 2013 07:33:55 +0000 (UTC) Received: (qmail 85331 invoked by uid 500); 17 Apr 2013 07:33:49 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 84994 invoked by uid 500); 17 Apr 2013 07:33:48 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 84974 invoked by uid 99); 17 Apr 2013 07:33:47 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Apr 2013 07:33:47 +0000 X-ASF-Spam-Status: No, hits=3.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_BRBL_LASTEXT,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [218.56.32.12] (HELO mail.haiyisoft.com) (218.56.32.12) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 17 Apr 2013 07:33:43 +0000 Received: from jccPC ([172.20.182.78]) (envelope-sender ) by 172.20.32.37 with ESMTP for ; Wed, 17 Apr 2013 15:32:58 +0800 From: "jiangchaocai" To: Cc: "'Henry Hung'" References: <53DC189E5FAEFA43BFA1BC02431031DB5FD5ABF6B8@WECTMLBOX.winbond.com.tw> <53DC189E5FAEFA43BFA1BC02431031DB5FD5ABF80F@WECTMLBOX.winbond.com.tw> In-Reply-To: Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_Adding_new_name_node_locatio?= =?UTF-8?Q?n?= Date: Wed, 17 Apr 2013 15:32:57 +0800 Message-ID: <000c01ce3b3d$c8554b80$58ffe280$@com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_000D_01CE3B80.D6788B80" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Ac47O8l/hjQ7Fb3dTfa/Myu3b1tXcQAAI5rwAABZoxA= Content-Language: zh-cn X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. ------=_NextPart_000_000D_01CE3B80.D6788B80 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable NOT authoritative: =20 The =E2=80=9Cnew name node=E2=80=9D will get the full copy of the name = node metadata when you restart the dfs service. During the hdfs running , same changes will be made on the name node and = the backup folder. In fact, it only contains 2 files: FSImage, EditLog =20 Thanks, John =20 =E5=8F=91=E4=BB=B6=E4=BA=BA: =E6=9D=8E=E6=B4=AA=E5=BF=A0 = [mailto:lhztop@hotmail.com]=20 =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2013=E5=B9=B44=E6=9C=8817=E6=97=A5 = 15:18 =E6=94=B6=E4=BB=B6=E4=BA=BA: user@hadoop.apache.org =E6=8A=84=E9=80=81: Henry Hung =E4=B8=BB=E9=A2=98: Re: Adding new name node location =20 modify conf file and restart name node is the best way. you needn't = restart the cluster DFS. the files in /backup and /home are the same. =E4=BA=8E 2013/4/17 14:38, Henry Hung =E5=86=99=E9=81=93: Hi Varun Kumar, =20 Could you be more elaborate about how the new changes being made to new = name node? =20 The scenario in my mind is: Suppose old name node metadata contains 100 hdfs files. Then I restart by using stop-dfs, change config and start-dfs. Hadoop will automatically create new name node directory in /backup. Now I have one old name node that have metadata of 100 hdfs files and = new name node that have none metadata. I put a new file into hdfs, the old name node will have 101 and new name = node will have 1. =20 Best regards, Henry =20 From: varun kumar [mailto:varun.uid@gmail.com]=20 Sent: Wednesday, April 17, 2013 2:18 PM To: user Cc: MA11 YTHung1 Subject: Re: Adding new name node location =20 Hi Henry, =20 As per your mail Point number 1 is correct. =20 After doing these changes metadata will be written in the new partition. =20 Regards, Varun Kumar.P =20 On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung = wrote: Hi Everyone, =20 I=E2=80=99m using Hadoop 1.0.4 and only define 1 location for name node = files, like this: dfs.name.dir /home/hadoop/hadoop-data/namenode =20 Now I want to protect my name node files by changing the configuration = to: dfs.name.dir = /home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namen= ode =20 Where /backup is another mount point. This /backup can be another disk = or from another NFS server. =20 My question are: 1. Is my procedure correct: do stop-dfs.sh then modify conf, and = last start-dfs.sh? 2. If answer to no 1 is no, then could you provide the correct = procedure? 3. Would the new name node files will auto copy the original name = node files? =20 Best regards, Henry =20 _____ =20 The privileged confidential information contained in this email is = intended for use only by the addressees as indicated by the original = sender of this email. If you are not the addressee indicated in this = email or are not responsible for delivery of the email to such a person, = please kindly reply to the sender indicating this fact and delete all = copies of it from your computer and network server immediately. Your = cooperation is highly appreciated. It is advised that any unauthorized = use of confidential information of Winbond is strictly prohibited; and = any information in this email irrelevant to the official business of = Winbond shall be deemed as neither given nor endorsed by Winbond. =20 --=20 Regards, Varun Kumar.P =20 _____ =20 The privileged confidential information contained in this email is = intended for use only by the addressees as indicated by the original = sender of this email. If you are not the addressee indicated in this = email or are not responsible for delivery of the email to such a person, = please kindly reply to the sender indicating this fact and delete all = copies of it from your computer and network server immediately. Your = cooperation is highly appreciated. It is advised that any unauthorized = use of confidential information of Winbond is strictly prohibited; and = any information in this email irrelevant to the official business of = Winbond shall be deemed as neither given nor endorsed by Winbond. =20 ------=_NextPart_000_000D_01CE3B80.D6788B80 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

NOT authoritative:

 

The =E2=80=9Cnew name node=E2=80=9D will get the full copy of the = name node metadata when you restart the dfs = service.

During the hdfs running , same changes will be made on the name node = and the backup folder.

In fact, it only contains 2 files: FSImage, = EditLog

 

Thanks,

John

 

=E5=8F=91=E4=BB=B6=E4=BA=BA: =E6=9D=8E=E6=B4=AA=E5=BF=A0 [mailto:lhztop@hotmail.com] =
=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2013=E5=B9=B44=E6=9C=8817=E6=97=A5 15:18
=E6=94=B6=E4=BB=B6=E4=BA=BA: user@hadoop.apache.org
=E6=8A=84=E9=80=81: Henry Hung
=E4=B8=BB=E9=A2=98: Re: Adding new name node = location

 

modify conf file and restart name = node is the best way. you needn't restart the cluster DFS.
the files = in /backup and /home are the same.
=E4=BA=8E 2013/4/17 14:38, Henry Hung =E5=86=99=E9=81=93:

Hi Varun Kumar,

 

Could  you be more elaborate about how the new changes being = made to new name node?

 

The scenario in my mind is:

Suppose old name node metadata contains 100 hdfs files.

Then I restart by using stop-dfs, change config and = start-dfs.

Hadoop will automatically create new name node directory in = /backup.

Now I have one old name node that have metadata of 100 hdfs files and = new name node that have none metadata.

I put a new file into hdfs, the old name node will have 101 and new = name node will have 1.

 

Best regards,

Henry

 

From:= varun = kumar [mailto:varun.uid@gmail.com] =
Sent: Wednesday, April 17, 2013 2:18 PM
To: = user
Cc: MA11 YTHung1
Subject: Re: Adding new name = node location

 

Hi = Henry,

 

As per your mail Point number 1 is = correct.

 

After doing these changes metadata = will be written in the new partition.

 

Regards,

Varun = Kumar.P

 

On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YTHung1@winbond.com> = wrote:

Hi Everyone,

 

I=E2=80=99m using Hadoop 1.0.4 and only define 1 location = for name node files, like this:

  <property>

    = <name>dfs.name.dir</name>

    = <value>/home/hadoop/hadoop-data/namenode</value>

  </property>

 

Now I want to protect my name node files by changing the = configuration to:

  <property>

    = <name>dfs.name.dir</name>

    = <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data= /namenode</value>

  </property>

 

Where /backup is another mount point. This /backup can be = another disk or from another NFS server.

 

My question are:

1.       = Is my procedure correct: do stop-dfs.sh then = modify conf, and last start-dfs.sh?

2.       = If answer to no 1 is no, then could you = provide the correct procedure?

3.       = Would the new name node files will auto copy = the original name node files?

 

Best regards,

Henry

 

The= privileged confidential information contained in this email is intended = for use only by the addressees as indicated by the original sender of = this email. If you are not the addressee indicated in this email or are = not responsible for delivery of the email to such a person, please = kindly reply to the sender indicating this fact and delete all copies of = it from your computer and network server immediately. Your cooperation = is highly appreciated. It is advised that any unauthorized use of = confidential information of Winbond is strictly prohibited; and any = information in this email irrelevant to the official business of Winbond = shall be deemed as neither given nor endorsed by Winbond.



 

--

Regards,

Varun = Kumar.P

 

The= privileged confidential information contained in this email is intended = for use only by the addressees as indicated by the original sender of = this email. If you are not the addressee indicated in this email or are = not responsible for delivery of the email to such a person, please = kindly reply to the sender indicating this fact and delete all copies of = it from your computer and network server immediately. Your cooperation = is highly appreciated. It is advised that any unauthorized use of = confidential information of Winbond is strictly prohibited; and any = information in this email irrelevant to the official business of Winbond = shall be deemed as neither given nor endorsed by Winbond.

 

------=_NextPart_000_000D_01CE3B80.D6788B80--