hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14365) Stabilise FileSystem builder-based create API
Date Sat, 29 Apr 2017 16:22:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989929#comment-15989929
] 

Steve Loughran commented on HADOOP-14365:
-----------------------------------------

I'm marking this as a blocker to make clear I don't consider the feature ready for use yet.
Core functionality behind the scenes may be good, a replacement for the massively overridden
{{create()}} needed, but without the specification and the set of cross-FS tests designed
to break it rather that just demo it, it's not something people can trust.

I think it wasn't ready for branch-2 yet, but as its there, it's there. I'm just adding this
as a blocker to say "people mustn't use the API yet", and to force the fixing of the problem,
rather than just people hoping to ignore this JIRA. Feel free to download for a to critical
release after first updating the release notes for HDFS-11170 to say "not ready for use, method
name may change, etc". 

> Stabilise FileSystem builder-based create API 
> ----------------------------------------------
>
>                 Key: HADOOP-14365
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14365
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Priority: Blocker
>
> HDFS-11170 added a builder-based create API for file creation which has a few issues
to work out before it can be considered ready for use
> 1. There no specification in the filesystem.md of what it is meant to do, which means
there's no public documentation on expected behaviour except on the Javadocs, which consists
of the sentences "Create a new FSDataOutputStreamBuilder for the file with path" and "Base
of specific file system FSDataOutputStreamBuilder".
> I propose:
> # Give the new method a relevant name rather than just define the return type, e.g. {{createFile()}}.

> # `Filesystem.md` to be extended with coverage of this method, and, sadly for the authors,
coverage of what the semantics of {{FSDataOutputStreamBuilder.build()}} are.
> 2. There are only tests for HDFS and local, neither of them perfect. Proposed: move to
{{AbstractContractCreateTest}}, test for all filesystems, fix tests and FS where appropriate.

> 3. Add more tests to generate the failure conditions implied by the updated filesystem
spec. Eg. create over a an existing file, create over a directory, create with negative buffer
size, negative block size, empty dest path, etc, etc. 
> This will clarify when precondition checks are made, as well as whether. For example:
should {{newFSDataOutputStreamBuilder()}} validate the path immediately?
> 4. Add to {{FileContext}}.
> 5. Take the opportunity to look at the flaws in today's {{create()}} calls and address
them, rather than replicate. In particular, I'd like to end the behaviour "create all parent
dirs.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message