hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8137) Sends the EC schema to DataNode as well in EC encoding/recovering command
Date Thu, 30 Apr 2015 13:04:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521460#comment-14521460
] 

Kai Zheng commented on HDFS-8137:
---------------------------------

Hi Uma,
bq.We supposed to get schema values from ECSchemaManager, but right now I don't see a better
way to get from ECScheaManeger, so I added an API to get from BlockCollection itself like
isStriped API in it.
{{ECSchemaManager}} might not be supposed to get a schema associated with a zone, dir/file,
but {{ErasureCodingZoneManager}} may do. We could query the schema info from a zone using
ErasureCodingZoneManager. I thought it's good to add the method {{getECSchema}} along with
the existing method {{isStriped}}, as it's essential to erasure coded files.
A quick look at the patch found it might need to align with some latest changes, regarding
how to get schema from a zone/dir/xAttr, would you double check? Thanks.

> Sends the EC schema to DataNode as well in EC encoding/recovering command
> -------------------------------------------------------------------------
>
>                 Key: HDFS-8137
>                 URL: https://issues.apache.org/jira/browse/HDFS-8137
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Uma Maheswara Rao G
>         Attachments: HDFS-8137-0.patch
>
>
> Discussed with [~umamaheswararao] and [~vinayrpet], we should also send the EC schema
to DataNode as well contained in the EC encoding/recovering command. The target DataNode will
use it to guide the executing of the task. 
> Another way would be, DataNode would just request schema actively thru a separate RPC
call, and as an optimization consideration, DataNode may cache schemas to avoid repeatedly
asking for the same schema twice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message