Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EF925DC4E for ; Tue, 11 Sep 2012 09:09:00 +0000 (UTC) Received: (qmail 71192 invoked by uid 500); 11 Sep 2012 09:08:56 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 70991 invoked by uid 500); 11 Sep 2012 09:08:55 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 70977 invoked by uid 99); 11 Sep 2012 09:08:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Sep 2012 09:08:55 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ramesh.narasingu@gmail.com designates 209.85.219.48 as permitted sender) Received: from [209.85.219.48] (HELO mail-oa0-f48.google.com) (209.85.219.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Sep 2012 09:08:49 +0000 Received: by oagn16 with SMTP id n16so138530oag.35 for ; Tue, 11 Sep 2012 02:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ADj9HfCZnaFb+w1cqO54q055EHxRD78yZaQIBZ6CF6w=; b=fTM9LJLVGCqExtrb3NbwLgGGc0YSpp/6yQX5Mt/u9/Q2qpEL+cX47gjEk2H1zklp6R p5/UjQP0UUUzHFwY1GImjtmfXXA6BaayR8bnnDab1v2IwU7pAiDEDXKOvtKwUosiNvRP pRfPjL9Zq4IGibuUvBW9ZmIbuXbx541IJRpShyNrvi5jJsnk0776sk28Ox88qKyRQtip fcbpJK59igRqMnoe+9LJ1YYfkPe7l8OD5L7K+wYwCStS1g2IiZXAi4+2/COjKBEQtMgC SXTBi4nyCJaJbQbHJ+bj3faplgiZFuOCHp5WXs9BlWpvA/vpfmR5iyoEW55JDgTIl1Lg MpYQ== MIME-Version: 1.0 Received: by 10.182.53.103 with SMTP id a7mr16951126obp.3.1347354508271; Tue, 11 Sep 2012 02:08:28 -0700 (PDT) Received: by 10.182.65.234 with HTTP; Tue, 11 Sep 2012 02:08:28 -0700 (PDT) In-Reply-To: References: <504EE772.5070909@jp.fujitsu.com> Date: Tue, 11 Sep 2012 14:38:28 +0530 Message-ID: Subject: Re: what happens when a datanode rejoins? From: Narasingu Ramesh To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=f46d04479631da210b04c9696a18 --f46d04479631da210b04c9696a18 Content-Type: text/plain; charset=ISO-8859-1 Hi Mehul, DataNode rejoins take care of only NameNode. Thanks & Regards, Ramesh.Narasingu On Tue, Sep 11, 2012 at 2:36 PM, Mehul Choube wrote: > > The namenode will asynchronously replicate the blocks to other > datanodes in order to maintain the replication factor after a datanode has > not been in contact for 10 minutes.**** > > What happens when the datanode rejoins after namenode has already > re-replicated the blocs it was managing?**** > > Will namenode ask the datanode to discard the blocks and start managing > new blocks?**** > > Or will namenode discard the new blocks which were replicated due to > unavailability of this datanode?**** > > ** ** > > ** ** > > ** ** > > Thanks,**** > > Mehul**** > > ** ** > > ** ** > > *From:* George Datskos [mailto:george.datskos@jp.fujitsu.com] > *Sent:* Tuesday, September 11, 2012 12:56 PM > *To:* user@hadoop.apache.org > *Subject:* Re: what happens when a datanode rejoins?**** > > ** ** > > Hi Mehul**** > > Some of the blocks it was managing are deleted/modified?**** > > > The namenode will asynchronously replicate the blocks to other datanodes > in order to maintain the replication factor after a datanode has not been > in contact for 10 minutes. > > > **** > > The size of the blocks are now modified say from 64MB to 128MB?**** > > > Block size is a per-file setting so new files will be 128MB, but the old > ones will remain at 64MB. > > > **** > > What if the block replication factor was one (yea not in most deployments > but say incase) so does the namenode recreate a file once the datanode > rejoins?**** > > > (assuming you didn't perform a decommission) Blocks that lived only on > that datanode will be declared "missing" and the files associated with > those blocks will be not be able to be fully read, until the datanode > rejoins. > > > > George**** > --f46d04479631da210b04c9696a18 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Mehul,
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 DataNode rejoins take car= e of only NameNode.
Thanks & Regards,
Ramesh.Narasingu

On Tue, Sep 11, 2012 at 2:36 PM, Mehul Choube <Mehul_Choube@symantec.com> wrote:

> The namenode will asynchronously repli= cate the blocks to other datanodes in order to maintain the replication fac= tor after a datanode has not been in contact for 10 minutes.<= /p>

What happens when the datanode rejoins after n= amenode has already re-replicated the blocs it was managing?<= /p>

Will namenode ask the datanode to discard the blo= cks and start managing new blocks?

Or will namenode discard the new blocks which were r= eplicated due to unavailability of this datanode?

=A0

=A0

=A0

Thanks,

Mehul

=A0

=A0<= /p>

From: George Datskos [mailto:george.datskos@jp.fujitsu.com]=
Sent: Tuesday, September 11, 2012 12:56 PM
To: user@hadoop.apache.org=
Subject: Re: what happens when a datanode rejoins?=

=A0=

Hi Mehu= l

Some of the blocks it was managing are deleted/modif= ied?


The namenode = will asynchronously replicate the blocks to other datanodes in order to mai= ntain the replication factor after a datanode has not been in contact for 1= 0 minutes.


The size of the blocks are= now modified say from 64MB to 128MB?


Block size is a per-file setting so new files will be 128MB, but the= old ones will remain at 64MB.


What if the block replicat= ion factor was one (yea not in most deployments but say incase) so does the= namenode recreate a file once the datanode rejoins?


(assuming you didn't perform a decommission) Blocks that lived only= on that datanode will be declared "missing" and the files associ= ated with those blocks will be not be able to be fully read, until the data= node rejoins.



George


--f46d04479631da210b04c9696a18--