hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8137) Sends the EC schema to DataNode as well in EC encoding/recovering command
Date Thu, 30 Apr 2015 07:02:06 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Uma Maheswara Rao G updated HDFS-8137:
    Attachment: HDFS-8137-0.patch

I generated an initial patch for review!
We supposed to get schema values from ECSchemaManager, but right now I don't see a better
way to get from ECScheaManeger, so I added an API to get from BlockCollection itself like
isStriped API in it. It's because BlockManager communicates with namesystem via Namesystem
interface. I don't think its right to add apis there for every new features. BlockCollection
is another interface like that and I added there. But logically Namesystem may be correct
place to add getECSchema for a file path . But I am not too strong on that. I would like hear
the suggestion on that if any.

> Sends the EC schema to DataNode as well in EC encoding/recovering command
> -------------------------------------------------------------------------
>                 Key: HDFS-8137
>                 URL: https://issues.apache.org/jira/browse/HDFS-8137
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Uma Maheswara Rao G
>         Attachments: HDFS-8137-0.patch
> Discussed with [~umamaheswararao] and [~vinayrpet], we should also send the EC schema
to DataNode as well contained in the EC encoding/recovering command. The target DataNode will
use it to guide the executing of the task. 
> Another way would be, DataNode would just request schema actively thru a separate RPC
call, and as an optimization consideration, DataNode may cache schemas to avoid repeatedly
asking for the same schema twice.

This message was sent by Atlassian JIRA

View raw message