hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7348) Erasure Coding: striped block recovery
Date Tue, 05 May 2015 14:00:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yi Liu updated HDFS-7348:
-------------------------
    Attachment: HDFS-7348.003.patch

Update the patch for Zhe's comments.
I also remove the hardcode of (dataBlkNum, parityBlkNum, cellSize), since they are now available
in ECSchema from BlockECRecoveryCommand.
The test passed in my local environment and current tests include recovering one or more parity
blocks and recovering  3 data blocks, we can update {{recoverTargets}} and related tests in
follow-on after the decode issue(HADOOP-11847) fixed.

The comments are addressed in the patch and some replies:
{quote}
I guess somewhere in the code (not necessarily in this path) we should assert all internal
blocks indeed share the same checksum?
{quote}
You are right, we can assert them, we can do it in a {{else}}.

{quote}
In the main run loop, success and success4Target look a little confusing ...
{quote}
Right, I rename {{success4Target}} to {{targetsStatus}} as you suggested, I keep others and
not use BitSet, since we need array access for them, which is more efficient and convenient
for code.

Besides, I add {{targetInputStreams}}, they are used in the sasl negotiation when connecting
to other DNs, and we'd better to close it too, even thought we had closed it's socket.

> Erasure Coding: striped block recovery
> --------------------------------------
>
>                 Key: HDFS-7348
>                 URL: https://issues.apache.org/jira/browse/HDFS-7348
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Kai Zheng
>            Assignee: Yi Liu
>         Attachments: ECWorker.java, HDFS-7348.001.patch, HDFS-7348.002.patch, HDFS-7348.003.patch
>
>
> This JIRA is to recover one or more missed striped block in the striped block group.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message