hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7782) Read a striping layout file from client side
Date Wed, 25 Mar 2015 18:38:53 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380457#comment-14380457
] 

Zhe Zhang commented on HDFS-7782:
---------------------------------

Thanks Jing for the quick feedback!

bq. We can first only provide a non-parallel and parallel reading of data blocks. 
Agreed. I think the 004 patch aligns with this plan. It has serial and parallel modes without
hedge or error handling.

bq. I think finally the client should be able to 1) read in parallel, and 2) handle slow datanodes
with data blocks, and 3) correctly handle possibly missing data blocks. Maybe this finally
should be all necessary, and users do not need to use configuration to turn it on.
This is a great summary. In this long term vision do you think serial read will still be a
mode to support? If so I guess we still need a switch to configure serial vs. parallel reads?

> Read a striping layout file from client side
> --------------------------------------------
>
>                 Key: HDFS-7782
>                 URL: https://issues.apache.org/jira/browse/HDFS-7782
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7782-000.patch, HDFS-7782-001.patch, HDFS-7782-002.patch, HDFS-7782-003.patch,
HDFS-7782-004.patch
>
>
> If client wants to read a file, he is not necessary to know and handle what layout the
file is. This sub task adds logic to DFSInputStream to support reading striping layout files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message