hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcos Ortiz <mlor...@uci.cu>
Subject Re: how does hdfs determine what node to use?
Date Thu, 10 Mar 2011 17:21:08 GMT
El 3/10/2011 8:37 AM, Rita escribió:
> Thanks Stu. I too was sure there was an algorithm. Is there a place 
> where I can read more about it?  I want to know if it picks a block 
> according to the load average or does it always pick "rack0" first?
>
>
>
> On Wed, Mar 9, 2011 at 10:24 PM, <stu24mail@yahoo.com 
> <mailto:stu24mail@yahoo.com>> wrote:
>
>     There is an algorithm. Each block should have a copy on different
>     nodes. In your case, each block will have a copy on each of the nodes.
>
>     Take care,
>     -stu
>     ------------------------------------------------------------------------
>     *From: * Rita <rmorgan466@gmail.com <mailto:rmorgan466@gmail.com>>
>     *Date: *Wed, 9 Mar 2011 22:07:37 -0500
>     *To: *<hdfs-user@hadoop.apache.org
>     <mailto:hdfs-user@hadoop.apache.org>>
>     *ReplyTo: * hdfs-user@hadoop.apache.org
>     <mailto:hdfs-user@hadoop.apache.org>
>     *Subject: *how does hdfs determine what node to use?
>
>     I have a 2 rack cluster. All of my files have a replication factor
>     of 2. How does hdfs determine what node to use when serving the
>     data? Does it always use the first rack? or is there an algorithm
>     for this?
>
>
>     -- 
>     --- Get your facts first, then you can distort them as you please.--
>
>
>
>
> -- 
> --- Get your facts first, then you can distort them as you please.--
The best source that I found for this is the Tom White´s book: Hadoop: 
The Definitive Guide 2nd Edition. Chapter 3: The Hadoop Distributed 
FileSystem, and the Hadoop wiki: http://wiki.apache.org/hadoop/HDFS.

Regards

-- 
Marcos Luís Ortíz Valmaseda
  Software Engineer
  Universidad de las Ciencias Informáticas
  Linux User # 418229

http://uncubanitolinuxero.blogspot.com
http://www.linkedin.com/in/marcosluis2186


Mime
View raw message