hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6803) Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent context
Date Wed, 13 Aug 2014 05:19:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095162#comment-14095162

Yi Liu commented on HDFS-6803:

It seems like we are all in agreement on Stack's point 2.1 (positional read and non-positional
can run concurrently), and 2.2 (two or more positional reads can run concurrently)

Should we require the implementations of {{FSInputStream}} should be thread-safe (or at least
for PositionedReadable)?  And modify some implementations such as WebHDFS to make pread concurrently?

There's always the strategy of adding a marker interface, say ConcurrentPositionalReads, which
indicates the operations are concurrent.
Maybe using existing PositionedReadable is OK and keep the concurrent in java doc as what
we do now?

> Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent
> --------------------------------------------------------------------------------------------
>                 Key: HDFS-6803
>                 URL: https://issues.apache.org/jira/browse/HDFS-6803
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: 2.4.1
>            Reporter: stack
>         Attachments: DocumentingDFSClientDFSInputStream (1).pdf
> Reviews of the patch posted the parent task suggest that we be more explicit about how
DFSIS is expected to behave when being read by contending threads. It is also suggested that
presumptions made internally be made explicit documenting expectations.
> Before we put up a patch we've made a document of assertions we'd like to make into tenets
of DFSInputSteam.  If agreement, we'll attach to this issue a patch that weaves the assumptions
into DFSIS as javadoc and class comments. 

This message was sent by Atlassian JIRA

View raw message