hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11644) DFSStripedOutputStream should not implement Syncable
Date Thu, 13 Apr 2017 19:22:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968113#comment-15968113

stack commented on HDFS-11644:

bq. "The current behavior of throwing an exception is safer."

... but changes precedent?

Semantics in general are messy here around sync, et al. It is a reflection of he torturous
journey taken by sync/flush/hflush/hsync in HDFS.

The blessed [~stevel@apache.org] tried writing a spec for DFS and got far on the read-side.
Helps. Write-side is to do. I like Steves' comment that rather than "....probe for interface,
cast, query, maintain.." at each point at which we encounter a feature, rather, there'd be
an upfront query that could be run before engaging w/ the fs implementation (though how does
this work if tiering changes the underlying storage on us at runtime?).

Meantime, having DFSStripedOutputStream throw an exception breaking all that run on top (with
no means of querying whether support or not) seems disruptive.

> DFSStripedOutputStream should not implement Syncable
> ----------------------------------------------------
>                 Key: HDFS-11644
>                 URL: https://issues.apache.org/jira/browse/HDFS-11644
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Andrew Wang
>            Assignee: Manoj Govindassamy
>              Labels: hdfs-ec-3.0-must-do
> FSDataOutputStream#hsync checks if a stream implements Syncable, and if so, calls hsync.
Otherwise, it just calls flush. This is used, for instance, by YARN's FileSystemTimelineWriter.
> DFSStripedOutputStream extends DFSOutputStream, which implements Syncable. However, DFSStripedOS
throws a runtime exception when the Syncable methods are called.
> We should refactor the inheritance structure so DFSStripedOS does not implement Syncable.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message