hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: Query regarding the replication factor in hadoop
Date Fri, 19 Sep 2014 14:55:40 GMT
Interesting. I thought that the write would fail in case if # of nodes
downs is greater than min-replication property. So in reality we only get a
warning while writing (and a info message through fsck.)

Regards,
Shahab

On Fri, Sep 19, 2014 at 9:26 AM, Abirami V <abiramipandian@gmail.com> wrote:

> You  will get under replicated block and missing replicas when you run
> hdfs fsck /
>
> you may see info like the following
>
> Under replicated blk_-4791859336845413240_1544. Target Replicas
> is 3 but found 2 replica(s).
>
>
>
> On Fri, Sep 19, 2014 at 5:36 AM, adarsh deshratnam <
> adarsh.deshratnam@gmail.com> wrote:
>
>> 1. *How hadoop will take care of balancing of replicas as the required
>> replicas are 3 , but we have only 2 data nodes up and running.*
>>
>> *Ans:* As here the replication factor is three. The data block will be
>> replicated three time within 2 nodes. Block replication is random.
>>
>> *2. What happens when we try to write new  data into hdfs at this point
>> of time ? whether the write would be successful with only 2 data nodes and
>> replication factor 3 or it returns any error message?*
>> *Ans:*It will write successfully.
>>
>>
>> For further info please refer below link:
>> http://hadoop.apache.org/docs/r1.2.1/hdfs_design.html
>>
>>
>> Thanks,
>> Adarsh D
>>
>> On Fri, Sep 19, 2014 at 5:46 PM, Raghavendra Chandra <
>> raghavchandra.learning@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I have one very basic query regarding the replication factor in HDFS.
>>>
>>> Scenario:
>>>
>>> I have 4 node cluster : 3 data nodes and 1 master node.
>>>
>>> The replication factor is 3. So ideally each data node would  get one
>>> replica .
>>>
>>> Assume that meanwhile one of the data node went down.
>>>
>>> so ideally we will be having 2 data nodes.
>>>
>>> Queries:
>>>
>>> 1. How hadoop will take care of balancing of replicas as the required
>>> replicas are 3 , but we have only 2 data nodes up and running.
>>>
>>> 2. What happens when we try to write new  data into hdfs at this point
>>> of time ? whether the write would be successful with only 2 data nodes and
>>> replication factor 3 or it returns any error message?
>>>
>>>
>>> These queries might be simple, but it would be really helpful if some
>>> one can answer.
>>>
>>> Thanks and regards,
>>> Raghav Chandra
>>>
>>>
>>
>

Mime
View raw message