hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6223) New improved FileSystem interface for those implementing new files systems.
Date Fri, 25 Sep 2009 18:41:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759685#action_12759685
] 

Sanjay Radia commented on HADOOP-6223:
--------------------------------------


Subject: Evolution of APIs: adding new arguments to create and rename in the future.

Take a look at AbstractFileSystem#create and AbstractFileSystem#createInternal.

The signature is
createInternal(final Path f, final EnumSet<CreateFlag> createFlag, CreateOpts... opts).


Original thought was that as we evolve and add new parameters the lower layers can ignore
the new arguments. 

This only works if the new arg is optional.
If the newly is manadatory then this does not work.

There 3 choices here

# ignore the problem - we will never add new mandatory args in the future
# add overloaded methods in the future as we add new args (classic approach)
# let the filesystem-impl throw unknownOpt with a list of unknown opts  and have the default
impl provide the required behaviour.

So choice 2 will be the familiar classic approach
{noformat}
final create(f, flag, CreateOpts ...opt) {
   // check arg validity
  // extract the arguments out of opts and add defaults for the missing ones
  createInternal(f, flag, permission, blocksize, checkSumSize, replication, iobuffersize,
progress);
}

abstract createInternal(f, flag, permission, blocksize, checkSumSize, replication, iobuffersize,
progress);
{noformat}

later when we add a new fooArg:
{noformat}
// filesystem-impl can override this if they desire better behaviour.
createInternal(f, flag, permission, blocksize, checkSumSize, replication, iobuffersize, progress,
fooArg) {
     // provide default behavious for argFoo
  createInternal(f, flag, permission, blocksize, checkSumSize, replication, iobuffersize,
progress);
}
{noformat}
   

Choice 3 will look like:

{noformat}
final create(f, flag, CreateOpts ...opt) {
   // check arg validity

  try {
  createInternal(f, flag, opt);
 } catch (e) {
   unknownOpt {
     // take the list of unknown options  and provide the default behaviour.
     // we are will have to deal with  default  impl for all new mandatory options
   } 
}

abstract createInternal((f, flag, CreateOpts ...opt);
{noformat}



> New improved FileSystem interface for those implementing new files systems.
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-6223
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6223
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Sanjay Radia
>            Assignee: Sanjay Radia
>         Attachments: AbstractFileSystem.java, Hdfs.java
>
>
> The FileContext API (HADOOP-4952) provides an improved interface for the application
writer.
> This lets us simplify the FileSystem API since it will no longer need to deal with notions
of default filesystem [ / ],  wd, and config
> defaults for blocksize, replication factor etc. Further it will not need the many overloaded
methods for create() and open() since
> the FileContext API provides that convenience.
> The FileSystem API can be simplified and can now be restricted to those implementing
new file systems.
> This jira proposes that we create new file system API,  and deprecate FileSystem API
after a few releases.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message