hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8731) Erasure coding: For a small file missing and under replicated ec-block calculation is incorrect
Date Wed, 08 Jul 2015 05:06:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617980#comment-14617980
] 

Walter Su commented on HDFS-8731:
---------------------------------

Is it duplicated with HDFS-8563?

> Erasure coding: For a small file missing and under replicated ec-block calculation is
incorrect 
> ------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8731
>                 URL: https://issues.apache.org/jira/browse/HDFS-8731
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: J.Andreina
>            Assignee: J.Andreina
>
> Write a small file (whose replica detail would be : one data replica and 3 parity replicas
.)
> Now :
> 	totalReplicasPerBlock = 4 
> 	targetFileReplication = 9 (calculated based on default RS-6-3 schema)
> Hence the small file ec-blocks is calculated as always missing and underreplicated. which
is wrong... 
> {code}
> targetFileReplication = (short) (ecSchema.getNumDataUnits() + ecSchema.getNumParityUnits());
> {code}
> {code}
>       // count missing replicas / under replicated blocks
>       if (totalReplicasPerBlock < targetFileReplication && totalReplicasPerBlock
> 0) {
>         res.missingReplicas += (targetFileReplication - totalReplicasPerBlock);
> 	res.numUnderReplicatedBlocks += 1;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message