hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6803) Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent context
Date Tue, 04 Nov 2014 05:26:34 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HDFS-6803:
    Attachment: DocumentingDFSClientDFSInputStream.v2.pdf

Need license to improve pread so back here again (pardon the absence).  

Attach v2 of the doc addressing nice feedback.  Dropped assertion 2.3 on filelength after
[~cmccabe]'s feedback above ("too much" and "too little") so now we are just assertions "2.1.
Positional read and non-positional read can run concurrently" and "2.2. Two or more positional
reads can run concurrently"

On adding a ConcurrentPositionalReads marker Interface, I'd say its probably too late for
this now; we've assumed we could do concurrent positional reads always and it sort of works
(slowly).  Would it be better marking those fs that do NOT support ConcurrentPositionalReads?
(Probably not -- too much work).

bq. I think stack's point 2.1 and 2.2 imply that pread can safely be called from multiple
threads concurrently. I guess we should document this too so that there's no confusion.

This is in the attached doc.  Will forward port it.

> Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent
> --------------------------------------------------------------------------------------------
>                 Key: HDFS-6803
>                 URL: https://issues.apache.org/jira/browse/HDFS-6803
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: 2.4.1
>            Reporter: stack
>         Attachments: DocumentingDFSClientDFSInputStream (1).pdf, DocumentingDFSClientDFSInputStream.v2.pdf
> Reviews of the patch posted the parent task suggest that we be more explicit about how
DFSIS is expected to behave when being read by contending threads. It is also suggested that
presumptions made internally be made explicit documenting expectations.
> Before we put up a patch we've made a document of assertions we'd like to make into tenets
of DFSInputSteam.  If agreement, we'll attach to this issue a patch that weaves the assumptions
into DFSIS as javadoc and class comments. 

This message was sent by Atlassian JIRA

View raw message