hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8320) Erasure coding: consolidate striping-related terminologies
Date Wed, 30 Sep 2015 19:35:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938407#comment-14938407
] 

Hudson commented on HDFS-8320:
------------------------------

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2408 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2408/])
HDFS-8320. Erasure coding: consolidate striping-related terminologies. (zhezhang: rev 7434c44b16765a49168c8f5cb01a3d0b817d158b)
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRecoverStripedFile.java
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPlanReadPortions.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestStripedBlockUtil.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java


> Erasure coding: consolidate striping-related terminologies
> ----------------------------------------------------------
>
>                 Key: HDFS-8320
>                 URL: https://issues.apache.org/jira/browse/HDFS-8320
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8320-HDFS-7285.00.patch, HDFS-8320-HDFS-7285.01.patch, HDFS-8320-HDFS-7285.02.patch,
HDFS-8320-HDFS-7285.03.patch
>
>
> Right now we are doing striping-based I/O in a number of places:
> # Client output stream (HDFS-7889)
> # Client input stream
> #* pread (HDFS-7782, HDFS-7678)
> #* stateful read (HDFS-8033, HDFS-8281, HDFS-8319)
> # DN reconstruction (HDFS-7348)
> In each place we use one or multiple of the following terminologies:
> # Cell
> # Stripe
> # Block group
> # Internal block
> # Chunk
> This JIRA aims to systematically define these terminologies in relation with each other
and in the context of the containing file. For example, a cell belong to stripe _i_ and internal
block _j_ can be indexed as {{(i, j)}} and its logical index _k_ in the file can be calculated.
> With the above consolidation, hopefully we can further consolidate striping I/O codes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message