hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8580) Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature
Date Mon, 15 Jun 2015 02:53:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Walter Su updated HDFS-8580:
----------------------------
    Summary: Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature
 (was: Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlockProto)

> Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-8580
>                 URL: https://issues.apache.org/jira/browse/HDFS-8580
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HDFS-8580-HDFS-7285.01.patch, HDFS-8580.00.patch
>
>
> Zhe Zhang, Kai Zheng and I had a offline discussion. Here is what we thought:  Add a
cellSize field in BlockInfoStriped as a workaround, and deal with memory usage in follow-on.(HDFS-8059)
> discussion in HDFS-8494:
> from Walter Su:
> {quote}
> I think BlockInfoStriped needs to keep cellSize.
> {quote}
> from [~vinayrpet]:
> {quote}
> I too was thinking the same when the FSImageLoader problem has came up. This will increase
the memory usage by ~4bytes for each block though.
> {quote}
> from [~jingzhao]
> {quote}
> Also, we should consider adding a chunk size field to StripedBlockProto and removing
the cell size field from HdfsFileStatus. In this way we can access the chunk size information
in the storage layer.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message