hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7782) Read a striping layout file from client side
Date Wed, 25 Mar 2015 18:29:55 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380435#comment-14380435
] 

Jing Zhao commented on HDFS-7782:
---------------------------------

Thanks for updating the patch, [~zhz]!

I think finally the client should be able to 1) read in parallel, and 2) handle slow datanodes
with data blocks, and 3) correctly handle possibly missing data blocks. Maybe this finally
should be all necessary, and users do not need to use configuration to turn it on. For #2
and #3, hedged read pattern will be necessary.

However, for our development, we can do it step by step. We can first only provide a non-parallel
and parallel reading of data blocks. Then as future work, we provide this hedged read functionality
and handle data recovery while reading.

> Read a striping layout file from client side
> --------------------------------------------
>
>                 Key: HDFS-7782
>                 URL: https://issues.apache.org/jira/browse/HDFS-7782
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7782-000.patch, HDFS-7782-001.patch, HDFS-7782-002.patch, HDFS-7782-003.patch,
HDFS-7782-004.patch
>
>
> If client wants to read a file, he is not necessary to know and handle what layout the
file is. This sub task adds logic to DFSInputStream to support reading striping layout files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message