Return-Path: 
 <user-return-123-apmail-hadoop-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 2FE889C62
	for <apmail-hadoop-user-archive@minotaur.apache.org>;
 Wed,  8 Aug 2012 17:31:48 +0000 (UTC)
Received: (qmail 84379 invoked by uid 500); 8 Aug 2012 17:31:43 -0000
Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org
Received: (qmail 84287 invoked by uid 500); 8 Aug 2012 17:31:43 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 84278 invoked by uid 99); 8 Aug 2012 17:31:43 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Aug 2012 17:31:43 +0000
X-ASF-Spam-Status: No, hits=2.2 required=5.0
	tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Received: from [209.85.212.48] (HELO mail-vb0-f48.google.com) (209.85.212.48)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Aug 2012 17:31:36 +0000
Received: by vbme21 with SMTP id e21so1091135vbm.35
        for <user@hadoop.apache.org>; Wed, 08 Aug 2012 10:31:14 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:x-gm-message-state;
        bh=DbkMQ3t+juOY/6ezE7SaTSO0JI8vgg0qDUtWH+eClMc=;
        b=J8jpeNcSaIOd8aMwHl1c5WQ8yXxUJIP79KZH5ZLalH7jRp17IKICiDFrce2deofl5r
         XSSPea+wHTqr8dJfqB67Wr2dG0W5DIK/u2ITdS6/F6SNu989xaCtTPUXECFCCcjWJeoy
         1ZCioH8XX5za0+Z0n6ln1Bo8L/5Nsu3kv9Be5qZHn6Eh3PVnycfZvhY8fZEPfVhY3l/6
         FyhkBogrYxv/aSJsy5soCKUnQ5Z1o+z6yyB89VKQY4dg7MVtfAa674/32v3bxJd2Vquq
         +UMioReICaGadwxQ9nGDla5ZGgVHeE+O1KqK4n8pncTmfMIVEs7HxHL4xOTd3nwbM7te
         2dVA==
MIME-Version: 1.0
Received: by 10.52.70.137 with SMTP id m9mr12621875vdu.49.1344447074134; Wed,
 08 Aug 2012 10:31:14 -0700 (PDT)
Received: by 10.52.21.209 with HTTP; Wed, 8 Aug 2012 10:31:14 -0700 (PDT)
In-Reply-To: <00bc01cd7585$4c585550$e508fff0$@com>
References: <00bc01cd7585$4c585550$e508fff0$@com>
Date: Wed, 8 Aug 2012 10:31:14 -0700
Message-ID: 
 <CA+4kjVuNWMRRn_fQrLMWc1HzvDY+Ggt4p+_xp_HRyqBCUMZsoQ@mail.gmail.com>
Subject: Re: is HDFS RAID "data locality" efficient?
From: Steve Loughran <stevel@hortonworks.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=bcaec5015e5545ec7804c6c47aaa
X-Gm-Message-State: 
 ALoCoQk6Il92u1nalKhfeoyzRDk68WdDbxtuR9U4cFSt5oiLMtcS35bLzcqHaqmPqto+ttiHD6X2

--bcaec5015e5545ec7804c6c47aaa
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 8 August 2012 09:46, Sourygna Luangsay <sluangsay@pragsis.com> wrote:

>  Hi folks!****
>
> One of the scenario I can think in order to take advantage of HDFS RAID
> without suffering this penalty is:**
>
> **-          **Using normal HDFS with default replication=3D3 for my =E2=
=80=9Cfresh
> data=E2=80=9D****
>
> **-          **Using HDFS RAID for my historical data (that is barely
> used by M/R)****
>
> ** **
>
>
>
exactly: less space use on cold data, with the penalty that access
performance can be worse. As the majority of data on a hadoop cluster is
usually "cold", it's a space and power efficient story for the archive data

--=20
Steve Loughran
Hortonworks Inc

--bcaec5015e5545ec7804c6c47aaa
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<br><br><div class=3D"gmail_quote">On 8 August 2012 09:46, Sourygna Luangsa=
y <span dir=3D"ltr">&lt;<a href=3D"mailto:sluangsay@pragsis.com" target=3D"=
_blank">sluangsay@pragsis.com</a>&gt;</span> wrote:<br><blockquote class=3D=
"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding=
-left:1ex">









<div lang=3D"ES" link=3D"blue" vlink=3D"purple">

<div>

<p class=3D"MsoNormal">Hi folks!<u></u><u></u></p>

<p class=3D"MsoNormal">One of the scenario I can think in order to
take advantage of HDFS RAID without suffering this penalty is:<u></u></p>

<p><u></u><span lang=3D"EN-US"><span>-<span style=3D"font:7.0pt &quot;Times=
 New Roman&quot;">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0
</span></span></span><u></u><span lang=3D"EN-US">Using normal HDFS with def=
ault
replication=3D3 for my =E2=80=9Cfresh data=E2=80=9D<u></u><u></u></span></p=
>

<p><u></u><span lang=3D"EN-US"><span>-<span style=3D"font:7.0pt &quot;Times=
 New Roman&quot;">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0
</span></span></span><u></u><span lang=3D"EN-US">Using HDFS RAID for my
historical data (that is barely used by M/R)<u></u><u></u></span></p>

<p class=3D"MsoNormal"><span lang=3D"EN-US"><u></u>=C2=A0<u></u></span></p>

<p class=3D"MsoNormal"><br></p></div></div></blockquote></div><div><br></di=
v><div>exactly: less space use on cold data, with the penalty that access p=
erformance can be worse. As the majority of data on a hadoop cluster is usu=
ally &quot;cold&quot;, it&#39;s a space and power efficient story for the a=
rchive data</div>
<div><br></div>-- <br><div>Steve Loughran</div><div>Hortonworks Inc</div><d=
iv><br></div>

--bcaec5015e5545ec7804c6c47aaa--