hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11644) DFSStripedOutputStream should not implement Syncable
Date Mon, 01 May 2017 13:39:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990806#comment-15990806
] 

Steve Loughran commented on HDFS-11644:
---------------------------------------

I'm going to link this to HDFS-11170: add a builder interface for creating files and HADOOP-14365,
make that builder interface ready for use, because i could imagine actually having the builder
aware of an option existing before its created. That is, maybe, you could probe for an FS
having an option before you build the output stream.

I'm not sure though; it may put too much intelligence into the builder, when the output stream
is the one which really knows what it has. If you want to query FS capabilities, e.g HBase
wants to make sure hsync works, it could create a stream, check the capabilities, fail if
unsupported.

> DFSStripedOutputStream should not implement Syncable
> ----------------------------------------------------
>
>                 Key: HDFS-11644
>                 URL: https://issues.apache.org/jira/browse/HDFS-11644
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Andrew Wang
>            Assignee: Manoj Govindassamy
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HDFS-11644.01.patch, HDFS-11644.02.patch
>
>
> FSDataOutputStream#hsync checks if a stream implements Syncable, and if so, calls hsync.
Otherwise, it just calls flush. This is used, for instance, by YARN's FileSystemTimelineWriter.
> DFSStripedOutputStream extends DFSOutputStream, which implements Syncable. However, DFSStripedOS
throws a runtime exception when the Syncable methods are called.
> We should refactor the inheritance structure so DFSStripedOS does not implement Syncable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message