Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 12 May 2015 12:16:00 +0000 (UTC)
From: "Yi Liu (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12829107.1431415381000.85443.1431432960144@Atlassian.JIRA>
In-Reply-To: <JIRA.12829107.1431415381000@Atlassian.JIRA>
References: <JIRA.12829107.1431415381000@Atlassian.JIRA>
 <JIRA.12829107.1431415381511@arcas>
Subject: [jira] [Updated] (HDFS-8376) Erasure Coding: Update last cellsize
 calculation according to whether the erasure codec has chunk boundary
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-8376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yi Liu updated HDFS-8376:
-------------------------
    Description: 
Current calculation for last cell size is as following. For parity cell, the last cell size is the same as the first data cell.  But some erasure codec has chunk boundary, then the last cellsize for parity block is the codec chunk size.
{code}
private static int lastCellSize(int size, int cellSize, int numDataBlocks,
      int i) {
    if (i < numDataBlocks) {
      // parity block size (i.e. i >= numDataBlocks) is the same as 
      // the first data block size (i.e. i = 0).
      size -= i*cellSize;
      if (size < 0) {
        size = 0;
      }
    }
    return size > cellSize? cellSize: size;
  }
{code}

  was:
Current calculation for last cell size is as following. For parity cell, the last cell size is the same as the first data cell.  But as discussed in HDFS-8347 and on-line meeting, some erasure codec is not linear, then the last cellsize for parity block is the codec chunk size.
{code}
private static int lastCellSize(int size, int cellSize, int numDataBlocks,
      int i) {
    if (i < numDataBlocks) {
      // parity block size (i.e. i >= numDataBlocks) is the same as 
      // the first data block size (i.e. i = 0).
      size -= i*cellSize;
      if (size < 0) {
        size = 0;
      }
    }
    return size > cellSize? cellSize: size;
  }
{code}

This JIRA also removes some duplicate code
{{StripedBlockUtil#constructStripedBlock}} and {{StripedBlockUtil#getStripedBlockLength}} are duplicated with some other methods, we can clean up.

        Summary: Erasure Coding: Update last cellsize calculation according to whether the erasure codec has chunk boundary  (was: Erasure Coding: Update last cellsize calculation according to whether the erasure codec is linear)

> Erasure Coding: Update last cellsize calculation according to whether the erasure codec has chunk boundary
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8376
>                 URL: https://issues.apache.org/jira/browse/HDFS-8376
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>
> Current calculation for last cell size is as following. For parity cell, the last cell size is the same as the first data cell.  But some erasure codec has chunk boundary, then the last cellsize for parity block is the codec chunk size.
> {code}
> private static int lastCellSize(int size, int cellSize, int numDataBlocks,
>       int i) {
>     if (i < numDataBlocks) {
>       // parity block size (i.e. i >= numDataBlocks) is the same as 
>       // the first data block size (i.e. i = 0).
>       size -= i*cellSize;
>       if (size < 0) {
>         size = 0;
>       }
>     }
>     return size > cellSize? cellSize: size;
>   }
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)