hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Atul Sikaria (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14765) AdlFsInputStream to implement CanUnbuffer
Date Mon, 18 Sep 2017 22:24:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170829#comment-16170829
] 

Atul Sikaria commented on HADOOP-14765:
---------------------------------------

What is the semantic expectation of unbuffer()? The Interface doc doesn't say much:
https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/fs/CanUnbuffer.html

Is reducing the buffer equivalent to flushing everything we have in memory, or is there more
to it? Note that the sockets themselves are managed by the connection pool in the underlying
HttpUrlConnection used - and not sure what it would buy anybody to close those sockets if
they are not used (and would probably hurt if the connections were to be opened again).

So for now we can do a simple implementation that does flush() or nothing, but just trying
to understand what else is desired.



> AdlFsInputStream to implement CanUnbuffer
> -----------------------------------------
>
>                 Key: HADOOP-14765
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14765
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/adl
>    Affects Versions: 2.8.0
>            Reporter: John Zhuge
>            Priority: Minor
>
> HBase relies on FileSystems implementing CanUnbuffer.unbuffer() to force input streams
to free up remote connections (HBASE-9393Link). This works for HDFS, but not elsewhere.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message