Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of arthur.hk.chan@gmail.com
 designates 209.85.214.171 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALBaq-O7ave2FtC0idLS-RFw-4X5ZxbFXn0jS7G7LJw+7Nvg3A@mail.gmail.com>
References: <CA760494-9D36-4DCF-8830-8A50894504D5@gmail.com>
	<CALBaq-O7ave2FtC0idLS-RFw-4X5ZxbFXn0jS7G7LJw+7Nvg3A@mail.gmail.com>
Date: Sun, 5 Apr 2015 13:39:01 +0800
Message-ID: 
 <CAFDNJ6j-ZMr9Xk+GcC1fELO9yiCbbWncbZ85ry3R+i+3uGD2=g@mail.gmail.com>
Subject: Re: How will Hadoop handle it when a datanode server with total
 hardware failure?
From: Arthur Chan <arthur.hk.chan@gmail.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a11c2ea344f4e750512f3999e

--001a11c2ea344f4e750512f3999e
Content-Type: text/plain; charset=UTF-8

Hi,

I use the default replication factor 3 here, the cluster has 10 nodes, each
of my datanode has 8 hard disks.  If one of the nodes is down because of
hardware failure, i.e. the 8 hard disks will no longer be available
immediately during the down time of this machine, does it mean that I will
have data lost? (8 hard disks >  3 replicated)

Or what would be the maximum number of servers that are allowed to be down
without data lost here?

Regards
Arthur

On Wednesday, December 17, 2014, Harshit Mathur <mathursharp@gmail.com>
wrote:

> Hi Arthur,
>
> In HDFS there will be block level replication, In case of total failure of
> a datanode the lost blocks will get under replicated hence the namenode
> will create copy of these under replicated blocks on some other datanode.
>
> BR,
> Harshit
>
> On Wed, Dec 17, 2014 at 11:35 AM, Arthur.hk.chan@gmail.com
> <javascript:_e(%7B%7D,'cvml','Arthur.hk.chan@gmail.com');> <
> arthur.hk.chan@gmail.com
> <javascript:_e(%7B%7D,'cvml','arthur.hk.chan@gmail.com');>> wrote:
>>
>> Hi,
>>
>> If each of  my datanode servers has 8 hard disks (a 10-node cluster) and
>> I use the default replication factor of 3, how will Hadoop handle it when a
>> datanode with total hardware failure suddenly?
>>
>> Regards
>> Arthur
>>
>
>
>
> --
> Harshit Mathur
>

--001a11c2ea344f4e750512f3999e
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi,=C2=A0<div><span style=3D"line-height:normal"><br></span></div><div><spa=
n style=3D"line-height:normal">I use the default replication factor 3 here,=
 the cluster has 10 nodes, each of my datanode has 8 hard disks.=C2=A0 If o=
ne of the nodes is down because of hardware failure, i.e. the 8 hard disks =
will no longer be available immediately during the down time of this machin=
e, does it mean that I will have data lost? (8 hard disks &gt; =C2=A03 repl=
icated)</span></div><div><span style=3D"line-height:normal"><br></span></di=
v><div><span style=3D"line-height:normal">Or what would be the maximum numb=
er of servers that are allowed to be down without data lost here?=C2=A0</sp=
an></div><div><span style=3D"line-height:normal"><br></span></div><div><spa=
n style=3D"line-height:normal">Regards</span></div><div><span style=3D"line=
-height:normal">Arthur<br></span><br>On Wednesday, December 17, 2014, Harsh=
it Mathur &lt;<a href=3D"mailto:mathursharp@gmail.com">mathursharp@gmail.co=
m</a>&gt; wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div>Hi=
 Arthur,<br><br></div><div>In HDFS there will be block level replication, I=
n case of total failure of a datanode the lost blocks will get under replic=
ated hence the namenode will create copy of these under replicated blocks o=
n some other datanode.<br><br></div><div>BR,<br></div><div>Harshit<br></div=
></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, De=
c 17, 2014 at 11:35 AM, <a href=3D"javascript:_e(%7B%7D,&#39;cvml&#39;,&#39=
;Arthur.hk.chan@gmail.com&#39;);" target=3D"_blank">Arthur.hk.chan@gmail.co=
m</a> <span dir=3D"ltr">&lt;<a href=3D"javascript:_e(%7B%7D,&#39;cvml&#39;,=
&#39;arthur.hk.chan@gmail.com&#39;);" target=3D"_blank">arthur.hk.chan@gmai=
l.com</a>&gt;</span> wrote:<blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
If each of=C2=A0 my datanode servers has 8 hard disks (a 10-node cluster) a=
nd I use the default replication factor of 3, how will Hadoop handle it whe=
n a datanode with total hardware failure suddenly?<br>
<br>
Regards<br>
<span><font color=3D"#888888">Arthur<br>
=C2=A0</font></span></blockquote></div><br clear=3D"all"><br>-- <br><div>Ha=
rshit Mathur</div>
</div>
</blockquote></div>

--001a11c2ea344f4e750512f3999e--