hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8120) Erasure coding: created util class to analyze striped block groups
Date Mon, 13 Apr 2015 14:11:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492433#comment-14492433
] 

Walter Su commented on HDFS-8120:
---------------------------------

I carefully read {{DFSStripedOutputStream.writeParityCellsForLastStripe()}}, the {{lastStripeSizePerBlock}}
(or you can call it "the last partial unfilled cell")  in data block and in parity block are
same. {{lastStripeSizePerBlock}} equals to the size of (partial or not) cell in the *first*
data block.
So I think the calculation is:
{code:java}
  public static long getSizeOfInternalBlock(long blkGroupSize, int cellSize,
                                            int dataBlkNum) {
    int numStripes = (int) (blkGroupSize / (cellSize * dataBlkNum));
    long blkSize = numStripes * cellSize;
    long lastStripeSize = blkGroupSize -
        numStripes * cellSize * dataBlkNum;
    long lastStripeSizePerBlock = Math.min(cellSize, lastStripeSize);
    blkSize += lastStripeSizePerBlock;
    return blkSize;
  }
{code}

> Erasure coding: created util class to analyze striped block groups
> ------------------------------------------------------------------
>
>                 Key: HDFS-8120
>                 URL: https://issues.apache.org/jira/browse/HDFS-8120
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8120.000.patch, HDFS-8120.001.patch
>
>
> The patch adds logic of calculating size of individual blocks in a striped block group.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message