hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Sasaki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8136) Client gets and uses EC schema when reads and writes a stripping file
Date Thu, 16 Apr 2015 02:43:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497482#comment-14497482
] 

Kai Sasaki commented on HDFS-8136:
----------------------------------

I have a question about how to obtain schema information from NN. 
{{DFSClient}} is assumed to use {{DFSStripedInputStream}} after opening file. Which is better
to pass {{ECSchema}} from {{DFSClient}} or to restore {{ECSchema}} inside {{DFSStripedInputStream}}
itself? The reason why I have this question is that {{DFSStripedInputStream}} has not reference
to {{ClientProtocol}} now. I think these requests to NN should be delegated to {{DFSClient}}
because it is already responsible to handle {{ClientProtocol}}. 
Thank you.

> Client gets and uses EC schema when reads and writes a stripping file
> ---------------------------------------------------------------------
>
>                 Key: HDFS-8136
>                 URL: https://issues.apache.org/jira/browse/HDFS-8136
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: HDFS-7285
>            Reporter: Kai Zheng
>            Assignee: Kai Sasaki
>
> Discussed with [~umamaheswararao] and [~vinayrpet], in client when reading and writing
a stripping file, it can invoke a separate call to NameNode to request the EC schema associated
with the EC zone where the file is in. Then the schema can be used to guide the reading and
writing. Currently it uses hard-coded values.
> Optionally, as an optimization consideration, client may cache schema info per file or
per zone or per schema name. We could add schema name in {{HdfsFileStatus}} for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message