hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3493) Invalidate excess corrupted blocks as long as minimum replication is satisfied
Date Thu, 12 Jun 2014 21:19:05 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Wang updated HDFS-3493:
------------------------------

    Summary: Invalidate excess corrupted blocks as long as minimum replication is satisfied
 (was: Invalidate corrupted blocks as long as minimum replication is satisfied)

> Invalidate excess corrupted blocks as long as minimum replication is satisfied
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-3493
>                 URL: https://issues.apache.org/jira/browse/HDFS-3493
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha, 2.0.5-alpha
>            Reporter: J.Andreina
>            Assignee: Juan Yu
>             Fix For: 2.5.0
>
>         Attachments: HDFS-3493.002.patch, HDFS-3493.003.patch, HDFS-3493.004.patch, HDFS-3493.patch
>
>
> replication factor= 3, block report interval= 1min and start NN and 3DN
> Step 1:Write a file without close and do hflush (Dn1,DN2,DN3 has blk_ts1)
> Step 2:Stopped DN3
> Step 3:recovery happens and time stamp updated(blk_ts2)
> Step 4:close the file
> Step 5:blk_ts2 is finalized and available in DN1 and Dn2
> Step 6:now restarted DN3(which has got blk_ts1 in rbw)
> From the NN side there is no cmd issued to DN3 to delete the blk_ts1 . But ask DN3 to
make the block as corrupt .
> Replication of blk_ts2 to DN3 is not happened.
> NN logs:
> ========
> {noformat}
> INFO org.apache.hadoop.hdfs.StateChange: BLOCK NameSystem.addToCorruptReplicasMap: duplicate
requested for blk_3927215081484173742 to add as corrupt on XX.XX.XX.XX:50276 by /XX.XX.XX.XX
because reported RWR replica with genstamp 1007 does not match COMPLETE block's genstamp in
block map 1008
> INFO org.apache.hadoop.hdfs.StateChange: BLOCK* processReport: from DatanodeRegistration(XX.XX.XX.XX,
storageID=DS-443871816-XX.XX.XX.XX-50276-1336829714197, infoPort=50275, ipcPort=50277, storageInfo=lv=-40;cid=CID-e654ac13-92dc-4f82-a22b-c0b6861d06d7;nsid=2063001898;c=0),
blocks: 2, processing time: 1 msecs
> INFO org.apache.hadoop.hdfs.StateChange: BLOCK* Removing block blk_3927215081484173742_1008
from neededReplications as it has enough replicas.
> INFO org.apache.hadoop.hdfs.StateChange: BLOCK NameSystem.addToCorruptReplicasMap: duplicate
requested for blk_3927215081484173742 to add as corrupt on XX.XX.XX.XX:50276 by /XX.XX.XX.XX
because reported RWR replica with genstamp 1007 does not match COMPLETE block's genstamp in
block map 1008
> INFO org.apache.hadoop.hdfs.StateChange: BLOCK* processReport: from DatanodeRegistration(XX.XX.XX.XX,
storageID=DS-443871816-XX.XX.XX.XX-50276-1336829714197, infoPort=50275, ipcPort=50277, storageInfo=lv=-40;cid=CID-e654ac13-92dc-4f82-a22b-c0b6861d06d7;nsid=2063001898;c=0),
blocks: 2, processing time: 1 msecs
> WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Not able to
place enough replicas, still in need of 1 to reach 1
> For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
> {noformat}
> fsck Report
> ===========
> {noformat}
> /file21:  Under replicated BP-1008469586-XX.XX.XX.XX-1336829603103:blk_3927215081484173742_1008.
Target Replicas is 3 but found 2 replica(s).
> .Status: HEALTHY
>  Total size:	495 B
>  Total dirs:	1
>  Total files:	3
>  Total blocks (validated):	3 (avg. block size 165 B)
>  Minimally replicated blocks:	3 (100.0 %)
>  Over-replicated blocks:	0 (0.0 %)
>  Under-replicated blocks:	1 (33.333332 %)
>  Mis-replicated blocks:		0 (0.0 %)
>  Default replication factor:	1
>  Average block replication:	2.0
>  Corrupt blocks:		0
>  Missing replicas:		1 (14.285714 %)
>  Number of data-nodes:		3
>  Number of racks:		1
> FSCK ended at Sun May 13 09:49:05 IST 2012 in 9 milliseconds
> The filesystem under path '/' is HEALTHY
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message