hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14747) S3AInputStream to implement CanUnbuffer
Date Tue, 05 Sep 2017 11:26:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153495#comment-16153495

Steve Loughran commented on HADOOP-14747:

Should we stub this out in Hadoop 3.1 for s3a, wasb & ADL just to keep hbase happy?

> S3AInputStream to implement CanUnbuffer
> ---------------------------------------
>                 Key: HADOOP-14747
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14747
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.1
>            Reporter: Steve Loughran
> HBase relies on FileSystems implementing {{CanUnbuffer.unbuffer()}} to force input streams
to free up remote connections (HBASE-9393). This works for HDFS, but not elsewhere.
> S3A input stream can implement {{CanUnbuffer.unbuffer()}} by closing the input stream
and relying on lazy seek to reopen it on demand.
> Needs
> * Contract specification of unbuffer. As in "who added a new feature to filesystems but
forgot to mention what it should do?"
> * Contract test for filesystems which declare their support. 
> * S3AInputStream to call {{closeStream()}} on a call to {{unbuffer()}}.
> * Test case

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message