hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14495) Add set options interface to FSDataOutputStreamBuilder
Date Wed, 07 Jun 2017 19:10:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041445#comment-16041445

Lei (Eddy) Xu commented on HADOOP-14495:

Copy [~steve_l]'s recommendations from HADOOP-14394.

As I've noted before I'd prefer use string values to set bool/numeric options.
Mybe we would need to be able to declare whether an option was "optional", or a mandatory
setting, eg.
builder = createFile(path)
builder.mandatory("encryption", true)
builder.opt("erasure-coding", false)
builder.opt("fadvise", "random")
builder.mandatory("hsync", true) // better support real hsync, not pretend to
lets client code be out of sync with what's shipping on their classpath.
stops clients needing to know exactly which FS client they've got. All you need to know is
there's a builder you can set things on. Setting an unsupported/not-yet-supported option isn't
a problem, it is if you make it mandatory.
allows clients to code against a stream without needing the relevant JARs on the CP at compile
time, or hard code for a specific FS.
I believe it could help some of the filtering filesystems return different clients from different
paths, without the client needing to try casting it to different instances to eventually get
one which supports the option game.
could also let filter filesystems inject their own options in between. I can't imagine they
would, but they could.

Lets discuss the designs of the set options interfaces here.

> Add set options interface to FSDataOutputStreamBuilder 
> -------------------------------------------------------
>                 Key: HADOOP-14495
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14495
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message