hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8137) Sends the EC schema to DataNode as well in EC encoding/recovering command
Date Thu, 30 Apr 2015 13:04:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521460#comment-14521460

Kai Zheng commented on HDFS-8137:

Hi Uma,
bq.We supposed to get schema values from ECSchemaManager, but right now I don't see a better
way to get from ECScheaManeger, so I added an API to get from BlockCollection itself like
isStriped API in it.
{{ECSchemaManager}} might not be supposed to get a schema associated with a zone, dir/file,
but {{ErasureCodingZoneManager}} may do. We could query the schema info from a zone using
ErasureCodingZoneManager. I thought it's good to add the method {{getECSchema}} along with
the existing method {{isStriped}}, as it's essential to erasure coded files.
A quick look at the patch found it might need to align with some latest changes, regarding
how to get schema from a zone/dir/xAttr, would you double check? Thanks.

> Sends the EC schema to DataNode as well in EC encoding/recovering command
> -------------------------------------------------------------------------
>                 Key: HDFS-8137
>                 URL: https://issues.apache.org/jira/browse/HDFS-8137
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Uma Maheswara Rao G
>         Attachments: HDFS-8137-0.patch
> Discussed with [~umamaheswararao] and [~vinayrpet], we should also send the EC schema
to DataNode as well contained in the EC encoding/recovering command. The target DataNode will
use it to guide the executing of the task. 
> Another way would be, DataNode would just request schema actively thru a separate RPC
call, and as an optimization consideration, DataNode may cache schemas to avoid repeatedly
asking for the same schema twice.

This message was sent by Atlassian JIRA

View raw message