hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Higgins <scythe...@gmail.com>
Subject Re: How to get Under-replicated blocks information [Location]
Date Wed, 27 Feb 2013 15:14:17 GMT
When you run FSCK, what options did you run it with? If you're using 
'hadoop fsck /' then you will typically see a lot of dots being output and 
amongst those dots you should see missing/corrupt or under-replicated 
notices. Here's an excerpt containing under replicated blocks, for an 
example:

....................................................................................................
.....................................................................................
/user/blank/.staging/job_201302211222_0011/libjars/common.jar:  Under 
replicated blk_-856998378388135111_3050830. Target Replicas is 10 but found 
9 replica(s).
...............
..
/user/blank/.staging/job_201302211222_0733/job.jar:  Under replicated 
blk_4441280820634984431_3150889. Target Replicas is 10 but found 9 
replica(s).
.......
/user/blank/.staging/job_201302211222_0741/job.jar:  Under replicated 
blk_6336347773734645693_3150957. Target Replicas is 10 but found 9 
replica(s).
..........
/user/blank/.staging/job_201302211222_0755/job.jar:  Under replicated 
blk_-1209937263563068132_3154083. Target Replicas is 10 but found 9 
replica(s).
.
/user/blank/.staging/job_201302211222_0756/job.jar:  Under replicated 
blk_-365467798112255961_3154084. Target Replicas is 10 but found 9 
replica(s).
................................................................................
....................................................................................................
....................................................................................................
........................................[...]

The paths you see in the example are the HDFS locations within the cluster. 
That should be the information you're looking for.

Does this help?

-Shawn

On Wednesday, February 27, 2013 4:17:18 AM UTC-6, Dhanasekaran Anbalagan 
wrote:
>
> Hi Guys,
>
> I am running three machine cluster, with replication factor 2, I got 
> problem in replica i changed to 2.
> after I ran fsck  i got  Under-replicated blocks: 71 (0.0034828386 %)
>
>  Total size: 105829415143 B
>  Total dirs: 9704
>  Total files: 2038873 (Files currently being written: 2)
>  Total blocks (validated): 2038567 (avg. block size 51913 B) (Total open 
> file blocks (not validated): 2)
>  Minimally replicated blocks: 2038567 (100.0 %)
>  Over-replicated blocks: 0 (0.0 %)
>  Under-replicated blocks: 71 (0.0034828386 %)
>  Mis-replicated blocks: 0 (0.0 %)
>  Default replication factor: 2
>  Average block replication: 1.9999995
>  Corrupt blocks: 0
>  Missing replicas: 71 (0.0017414198 %)
>  Number of data-nodes: 2
>  Number of racks: 1
> FSCK ended at Wed Feb 27 05:12:31 EST 2013 in 32647 milliseconds
>
>
> Please guide what the block location of the 71. in HDFS file system.
>
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.
>  

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message