hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8720) Minimally replicated blocks counting from fsck is misleading
Date Tue, 07 Jul 2015 11:03:07 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Walter Su updated HDFS-8720:
----------------------------
    Attachment: HDFS-8720.01.patch

calculation of 01 patch:
replicas < minReplication ( UNDER MIN REPL'D BLOCKS )(HDFS-7537)
replicas == minReplication ( Minimally replicated blocks )
replicas < ReplicationFactor ( Under-replicated blocks )
replicas == ReplicationFactor ( Normally replicated blocks )
replicas > ReplicationFactor ( Over-replicated blocks )

if ReplicationFactor equals to minReplication, the block is counted by both Minimally and
Normally blocks.

*scenario*
Assume minReplication=1.
Assume I have a block blk_1001 with factor=2 and 2 replicas. It's Normally replicated but
not Minimally replicated.
Assume I have a block blk_1002 with factor=1 and 1 replicas. It's Normally replicated also
Minimally replicated. 
The output with 01 patch:
{noformat}
 Total blocks (validated):  2 (avg. block size 1024 B)
 Minimally replicated blocks:   1 (50.0 %)
 Normally replicated blocks:    2 (100.0 %)
 Over-replicated blocks:    0 (0.0 %)
 Under-replicated blocks:   0 (0.0 %)
 Mis-replicated blocks:     0 (0.0 %)
 Default replication factor:    2
 Average block replication: 1.5
 Missing blocks:        0
 Corrupt blocks:        0
 Missing replicas:      0 (0.0 %)
 Number of data-nodes:      2
{noformat}
Now I know one block(blk_1002) is in danger.

> Minimally replicated blocks counting from fsck is misleading
> ------------------------------------------------------------
>
>                 Key: HDFS-8720
>                 URL: https://issues.apache.org/jira/browse/HDFS-8720
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Walter Su
>            Assignee: Walter Su
>            Priority: Minor
>         Attachments: HDFS-8720.01.patch
>
>
> {noformat}
>  Total blocks (validated):      1 (avg. block size 17087 B)
>  Minimally replicated blocks:   1 (100.0 %)
>  Over-replicated blocks:        0 (0.0 %)
>  Under-replicated blocks:       0 (0.0 %)
>  Mis-replicated blocks:         0 (0.0 %)
>  Default replication factor:    3
>  Average block replication:     3.0
>  Missing blocks:                0
>  Corrupt blocks:                0
>  Missing replicas:              0 (0.0 %)
>  Number of data-nodes:          3
>  Number of racks:               1
> {noformat}
> "Minimally replicated blocks" actually means "*at least* Minimally replicated blocks"
here.
> I want to know how many blocks are in danger, whose number of replicas is *equals* to
{{minReplication}}. I can't get it from fsck.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message