hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amr Awadallah <...@cloudera.com>
Subject Re: The mechanism of choosing target datanodes
Date Thu, 23 Apr 2009 20:56:36 GMT
yes, it will be split across many nodes, and if possible each block will 
get a different datanode.

see following link for more details:

http://hadoop.apache.org/core/docs/current/hdfs_design.html#Data+Organization

-- amr

Alex Loddengaard wrote:
> I believe the blocks will be distributed across data nodes and not local to
> only one data node.  If this wasn't the case, then running a MR job on the
> file would only be local to one task tracker.
>
> Alex
>
> On Thu, Apr 23, 2009 at 2:14 AM, Xie, Tao <xietao1981@gmail.com> wrote:
>
>   
>> If a cluster has many datanodes and I want to copy a large file into DFS.
>> If the replication number is set to 1, does the namenode will put the file
>> data on one datanode or several nodes? I wonder if the file will be split
>> into blocks then different unique blocks are on different datanodes.
>>
>> --
>> View this message in context:
>> http://www.nabble.com/The-mechanism-of-choosing-target-datanodes-tp23193235p23193235.html
>> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>>
>>
>>     
>
>   

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message