hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8376) Erasure Coding: Update last cellsize calculation according to whether the erasure codec is linear
Date Tue, 12 May 2015 07:24:01 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yi Liu updated HDFS-8376:
-------------------------
    Description: 
Current calculation for last cell size is as following. For parity cell, the last cell size
is the same as the first data cell.  But as discussed in HDFS-8347 and on-line meeting, some
erasure codec is not linear, then the last cellsize for parity block is the codec chunk size.
{code}
private static int lastCellSize(int size, int cellSize, int numDataBlocks,
      int i) {
    if (i < numDataBlocks) {
      // parity block size (i.e. i >= numDataBlocks) is the same as 
      // the first data block size (i.e. i = 0).
      size -= i*cellSize;
      if (size < 0) {
        size = 0;
      }
    }
    return size > cellSize? cellSize: size;
  }
{code}

This JIRA also removes some duplicate code
{{StripedBlockUtil#constructStripedBlock}} and {{StripedBlockUtil#getStripedBlockLength}}
are duplicated with some other methods, we can clean up.

  was:
Current calculation for last cell size is as following. For parity cell, the last cell size
is the same as the first data cell.  But as discussed in HDFS-8347 and on-line meeting, some
erasure codec is not liner, then the last cellsize for parity block is the codec chunk size.
{code}
private static int lastCellSize(int size, int cellSize, int numDataBlocks,
      int i) {
    if (i < numDataBlocks) {
      // parity block size (i.e. i >= numDataBlocks) is the same as 
      // the first data block size (i.e. i = 0).
      size -= i*cellSize;
      if (size < 0) {
        size = 0;
      }
    }
    return size > cellSize? cellSize: size;
  }
{code}

This JIRA also removes some duplicate code
{{StripedBlockUtil#constructStripedBlock}} and {{StripedBlockUtil#getStripedBlockLength}}
are duplicated with some other methods, we can clean up.


> Erasure Coding: Update last cellsize calculation according to whether the erasure codec
is linear
> -------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8376
>                 URL: https://issues.apache.org/jira/browse/HDFS-8376
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>
> Current calculation for last cell size is as following. For parity cell, the last cell
size is the same as the first data cell.  But as discussed in HDFS-8347 and on-line meeting,
some erasure codec is not linear, then the last cellsize for parity block is the codec chunk
size.
> {code}
> private static int lastCellSize(int size, int cellSize, int numDataBlocks,
>       int i) {
>     if (i < numDataBlocks) {
>       // parity block size (i.e. i >= numDataBlocks) is the same as 
>       // the first data block size (i.e. i = 0).
>       size -= i*cellSize;
>       if (size < 0) {
>         size = 0;
>       }
>     }
>     return size > cellSize? cellSize: size;
>   }
> {code}
> This JIRA also removes some duplicate code
> {{StripedBlockUtil#constructStripedBlock}} and {{StripedBlockUtil#getStripedBlockLength}}
are duplicated with some other methods, we can clean up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message