hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "J.Andreina (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-8731) Erasure coding: For a small file missing and under replicated ec-block calculation is incorrect
Date Wed, 08 Jul 2015 05:28:04 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

J.Andreina resolved HDFS-8731.
------------------------------
    Resolution: Duplicate

> Erasure coding: For a small file missing and under replicated ec-block calculation is
incorrect 
> ------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8731
>                 URL: https://issues.apache.org/jira/browse/HDFS-8731
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: J.Andreina
>            Assignee: J.Andreina
>
> Write a small file (whose replica detail would be : one data replica and 3 parity replicas
.)
> Now :
> 	totalReplicasPerBlock = 4 
> 	targetFileReplication = 9 (calculated based on default RS-6-3 schema)
> Hence the small file ec-blocks is calculated as always missing and underreplicated. which
is wrong... 
> {code}
> targetFileReplication = (short) (ecSchema.getNumDataUnits() + ecSchema.getNumParityUnits());
> {code}
> {code}
>       // count missing replicas / under replicated blocks
>       if (totalReplicasPerBlock < targetFileReplication && totalReplicasPerBlock
> 0) {
>         res.missingReplicas += (targetFileReplication - totalReplicasPerBlock);
> 	res.numUnderReplicatedBlocks += 1;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message