Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of vin.boucher@gmail.com
 designates 74.125.82.48 as permitted sender)
From: Vincent Boucher <vin.boucher@gmail.com>
To: hdfs-user@hadoop.apache.org
Subject: Re: hdfs file - datanode association
Date: Tue, 11 Oct 2011 10:08:32 +0200
User-Agent: KMail/1.9.10
References: <201110061119.20723.vin.boucher@gmail.com>
 <20111006131212.GO5600@burko.lfod.us>
In-Reply-To: <20111006131212.GO5600@burko.lfod.us>
MIME-Version: 1.0
Content-Type: text/plain;
  charset="iso-8859-15"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Message-Id: <201110111008.32494.vin.boucher@gmail.com>

It seems there's such a solution in 0.21, let's see how it integrates with=
=20
0.20.203

https://issues.apache.org/jira/browse/HDFS-385
http://hadoopblog.blogspot.com/2009/09/hdfs-block-replica-placement-in-your=
=2Ehtml

Do you have any hint about the release date of the stable 0.21th version?

Vincent

Le Thursday 06 October 2011, Will Maier a =E9crit=A0:
> Hi Vincent-
>
> On Thu, Oct 06, 2011 at 11:19:20AM +0200, Vincent Boucher wrote:
> > We are wondering if it is possible to require that the namenode redirec=
ts
> > the blocks of the files of a given directory to a particular set of
> > datanodes?
> >
> > Our case is the following:
> >
> >  - Servers
> >     10 x mass storage servers of each 50TB, RAID6
> >        -> 500TB available for hdfs
> >     30 x working nodes with 6TB (no RAID)
> >        -> 180TB available for hdfs
> >
> > We'd like that the files stored in
> >    /hdfs/ms
> > be hosted on the mass storage (ms) servers,
> > while the files in
> >   /hdfs/wn
> > be hosted on the working nodes (wn).
>
> Why do you want to partition your datanodes this way? In our cluster,
> datanode size ranges from ~1 TB (no RAID) up to ~80TB (RAID). While in an
> ideal world all datanodes would be similar in configuration, we have not
> observed any issues with this arrangement in production.