hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile
Date Tue, 07 Apr 2015 06:15:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14482661#comment-14482661
] 

Jing Zhao commented on HDFS-8058:
---------------------------------

bq. In my thought, this would never happen, although the type is BlockInfo[], the actual instance
type is BlockInfoContiguous[] or BlockInfoStriped[], and we also do assertBlock check before
setBlock, addBlock, etc.

Yes, I understand the current patch can avoid mixed block types. But my concern is that the
way depending on the logic hidden in the code, instead of the explicit data type, may be error
prone (especially for other developers). Thus from the API point of view, currently I still
prefer declaring {{BlockInfoContiguous[]}} and {{BlockInfoStriped[]}} separately.

> Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-8058
>                 URL: https://issues.apache.org/jira/browse/HDFS-8058
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: HDFS-7285
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-8058.001.patch, HDFS-8058.002.patch
>
>
> This JIRA is to use {{BlockInfo[] blocks}} for both striped and contiguous blocks in
INodeFile.
> Currently {{FileWithStripedBlocksFeature}} keeps separate list for striped blocks, and
the methods there duplicate with those in INodeFile, and current code need to judge {{isStriped}}
then do different things. Also if file is striped, the {{blocks}} in INodeFile occupy a reference
memory space.
> These are not necessary, and we can use the same {{blocks}} to make code more clear.
> I keep {{FileWithStripedBlocksFeature}} as empty for follow use: I will file a new JIRA
to move {{dataBlockNum}} and {{parityBlockNum}} from *BlockInfoStriped* to INodeFile, since
ideally they are the same for all striped blocks in a file, and store them in block will waste
NN memory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message