hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Number of Under-Replicated Blocks ?
Date Mon, 20 Feb 2012 06:21:28 GMT
Hi,

The tool "hadoop fsck" will tell you which files are under replicated
with a count of what was expected instead, just run it over /.

While it isn't a 'normal' thing to see it come up suddenly it is still
in the safe zone, and is most likely an indicator that either one of
your DN or one of its disks has gone bad, or you have a bad
mapred.submit.replication value for your cluster size (default is 10
replicas for all MR job submit data), or bit rot of existing blocks on
HDDs around the cluster, etc. -- You can mostly spot the pattern of
files causing it by running the fsck and obtaining the listing.

On Mon, Feb 20, 2012 at 11:43 AM, praveenesh kumar <praveenesh@gmail.com> wrote:
> Hi,
>
> I am suddenly seeing some under-replicated blocks on my cluster. Although
> its not causing any problems, but It seems like few blocks are not
> replicated properly.
>
> Number of Under-Replicated Blocks     :     147
>
> Is it okay behavior on hadoop. If no, How can I know what are the files
> with under-replicated blocks and how can I configure it properly to reduce
> the number of under-replicated blocks.
>
> Thanks,
> Praveenesh



-- 
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

Mime
View raw message