hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8137) Sends the EC schema to DataNode as well in EC encoding/recovering command
Date Thu, 30 Apr 2015 14:44:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521585#comment-14521585

Uma Maheswara Rao G commented on HDFS-8137:

Thanks a lot for the review Kai. 
Good catch. You are right, we are storing in xattrs along with zone. 

ECSchemaManager might not be supposed to get a schema associated with a zone, dir/file, but
ErasureCodingZoneManager may do.
By mistake I said as ECSchemaManager. Your are right, I should have said as ErasureCodingZoneManager
as it has that related code what I was talking.

Also I added the getECSchema API in namesystem itself as we have already added some ECSchema
related API in FSNameSystem.  For reusing the codes from ECZoneManager codes, keeping this
new API in namesystem would give us the flexibility, but we can not get the same flexibility
from BlockCollection as we can not access FSDirectory details there. 

Please check if the latest patch make sense for you?

> Sends the EC schema to DataNode as well in EC encoding/recovering command
> -------------------------------------------------------------------------
>                 Key: HDFS-8137
>                 URL: https://issues.apache.org/jira/browse/HDFS-8137
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Uma Maheswara Rao G
>         Attachments: HDFS-8137-0.patch, HDFS-8137-1.patch
> Discussed with [~umamaheswararao] and [~vinayrpet], we should also send the EC schema
to DataNode as well contained in the EC encoding/recovering command. The target DataNode will
use it to guide the executing of the task. 
> Another way would be, DataNode would just request schema actively thru a separate RPC
call, and as an optimization consideration, DataNode may cache schemas to avoid repeatedly
asking for the same schema twice.

This message was sent by Atlassian JIRA

View raw message