hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13327) Add OutputStream + Syncable to the Filesystem Specification
Date Mon, 21 Aug 2017 11:29:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HADOOP-13327:
------------------------------------
    Attachment: HADOOP-13327-002.patch

HADOOP-13327 OutputStream. Syncable and StreamCapabilities

* The docs state that you must rely on StreamCapabilities.hasCapability() as the cue as to
whether Syncable methods are supported; tests are there to see what goes on The Azure and
ADL streams now support this interface.
* Issues with FSOutputSummer are documented. It needs to make close() at-most-once idempotent
& check for it being closed in write(int); explicitly downgrade flush() to no-op.
* Object store semantics listed

I don't think we've looked at `FSOutputSummer` hard enough to see what it does wrong, not
for a while. This does need fixing, but I'd rather delay that until Hadoop 3.1; something,
somewhere, will be using it wrong.

> Add OutputStream + Syncable to the Filesystem Specification
> -----------------------------------------------------------
>
>                 Key: HADOOP-13327
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13327
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-13327-002.patch, HADOOP-13327-branch-2-001.patch
>
>
> Write down what a Filesystem output stream should do. While core the API is defined in
Java, that doesn't say what's expected about visibility, durability, etc —and Hadoop Syncable
interface is entirely ours to define.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message