hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6450) Support non-positional hedged reads in HDFS
Date Fri, 06 Mar 2015 20:48:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350869#comment-14350869

Colin Patrick McCabe commented on HDFS-6450:

I think it would be possible to support hedged non-positional reads in {{BlockReaderLocal}},
but difficult.  First we would have to stop re-using the same FD for all instances of a BlockReaderLocal
that were reading the same replica.  Perhaps we could use dup to create a new FD per blockreader
without doing multiple opens.  Then we could close the blockreader FD if the local read were
being slow.

I think it's much easier to just implement hedged non-positional reads in the erasure coding-specific
subclass of DFSInputStream.

I also think we may want to create a base class for DFSInputStream that both the raid and
the non-raid code path inherit from.  Inheriting from the non-raid code path is weird because
there is a lot of stuff that is not relevant.

> Support non-positional hedged reads in HDFS
> -------------------------------------------
>                 Key: HDFS-6450
>                 URL: https://issues.apache.org/jira/browse/HDFS-6450
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.4.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Liang Xie
>         Attachments: HDFS-6450-like-pread.txt
> HDFS-5776 added support for hedged positional reads.  We should also support hedged non-position
reads (aka regular reads).

This message was sent by Atlassian JIRA

View raw message