hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ilayaraja" <ilayar...@rediff.co.in>
Subject Re: Problem: Some blocks remain under replicated
Date Tue, 31 Mar 2009 05:43:55 GMT
We are using hadoop-0.15.
Let me explain the scenario:
  We have around 6 TB of data in our cluster on couple of data 
directories(/mnt, /mnt2)with a replication factor of 1. when we increased 
the replication to 2 for the entire data, we observed that /mnt is used 100% 
while /mnt2 is under utilized. So we wanted to balance the utilization of 
space from both the data directories by changing the hadoop code for 
getNextVolume(..) API. The new algorithm checks which data directory is 
having more space available and returns that as the volume for the block to 
be written. This updated version of hadoop is then used for setting the 
replication to 2 for the entire dfs. However, when the dfs replication is 
over, it reported more than 100 GB of data blocks are missing as well as 
some blocks are under replicated. We also observed that there are many 
blocks of size zero present in the cluster, we do not know how these blocks 
were created.

----- Original Message ----- 
From: "Hairong Kuang" <hairong@yahoo-inc.com>
To: "hadoop-dev" <core-dev@hadoop.apache.org>; "ilayaraja" 
<ilayaraja@rediff.co.in>; "hadoop-user" <core-user@hadoop.apache.org>
Sent: Tuesday, March 31, 2009 3:30 AM
Subject: Re: Problem: Some blocks remain under replicated

> Which version of HADOOP are you running? Your cluster might have hit
> HADOOP-5465.
> Hairong
> On 3/29/09 10:24 PM, "ilayaraja" <ilayaraja@rediff.co.in> wrote:
>> Hello !
>> I am trying to increase the replication factor of a directory in our 
>> hadoop
>> dfs from 1 to 2.
>> I observe that some of the blocks (12 out of 400) always remain under
>> replicated, throwing the following message when I do an 'fsck' :
>>                     Under replicated blk_9084408236031628003. Target 
>> Replicas
>> is 2 but found 1 replica(s).
>> I thought it could be a problem with a specific data node in the cluster,
>> however I observe that the under replicated blocks belong to different 
>> data
>> nodes .
>> Please give me your thoughts.
>> Thanks.
>> Ilay

View raw message