hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14398) Modify documents for the FileSystem Builder API
Date Thu, 13 Jul 2017 21:49:03 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16086461#comment-16086461
] 

Aaron Fabbri commented on HADOOP-14398:
---------------------------------------

Hi [~eddyxu].  Thank you for working on this documentation, it looks pretty good.  Some comments
below.  I'm new to builder work so apologies if this has already been discussed.

{noformat}
+change on filesystem immediately. When `build()` be invoked on the `FSDataOutputStreamBuilder`,
+the builder parameters are verified and filesystem are modified. `build()` has
{noformat}

I think you mean the Filesystem instance; some filesystems don't modify the underlying FS
until close().  Would this be clearer as:

"When `build()` is invoked on the `FSDataOutputStreamBuilder`, the builder parameters are
verified and create() is invoked on the underlying filesystem."

You could also make a similar change to the following text on append, e.g.

"When `build()` is invoked on the `FSDataOutputStreamBuilder`, the builder parameters are
verified and append() is invoked on the underlying filesystem."

{noformat}
### Set optional or mandatory parameters
+
+    FSDataOutputStreamBuilder opt(String key, ...)
+    FSDataOutputStreamBuilder must(String key, ...)
+
+Set optional or mandatory parameters to the builder. Using `opt()` or `must()`,
+client can specify FS-specific parameters without inspecting the concrete type
+of `FileSystem`.
{noformat}

What is the difference in behavior / semantics between opt() and must()?

What is the conflict resolution behavior if there is a builder method (e.g. {{.someOption()}})
that conflicts with a {{Configuration}} option set via opt() or must()? ("undefined" is a
possibility, but should be specified at least?).

{noformat}
+The concrete `FileSystem` and/or `FSDataOutputStreamBuilder` implementation
+MUST verify that implementation-agnostic parameters (i.e., "syncable") or
+implementation-specific parameters ("fs.hdfs.lazy-persist" or "fs.s3a.fast-upload")
+are supported.
{noformat}

Even for {{opt()}}? 

{noformat}
+The following combinations of parameters are not supported:
+
+    if APPEND|OVERWRITE: raise HadoopIllegalArgumentException
+    if CREATE|APPEND|OVERWRITE: raise HadoopIllegalArgumentExdeption
{noformat}

Seems like the invalid settings are:
APPEND|OVERWRITE
CREATE|APPEND
CREATE|OVERWRITE

Or we could say "APPEND, OVERWRITE, and CREATE are mutually-exclusive: Setting more than one
of these flags will fail with HadoopIllegalArgumentException"?

We could also clarify that "The Builder does not validate that create flag combinations are
valid, but the underlying filesystem will fail on build() in the following cases:"

> Modify documents for the FileSystem Builder API
> -----------------------------------------------
>
>                 Key: HADOOP-14398
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14398
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>              Labels: docuentation
>         Attachments: HADOOP-14398.00.patch
>
>
> After finishes the API, we should update the document to describe the interface, capability
and contract which the APIs hold. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message