hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12910) Add new FileSystem API to support asynchronous method calls
Date Fri, 11 Mar 2016 21:53:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191541#comment-15191541
] 

Tsz Wo Nicholas Sze commented on HADOOP-12910:
----------------------------------------------

bq. I'm going to be ruthless and say "I'd like to see a specification of this alongside the
existing one". ...

This is a good idea so that we can make the API rigorous.

bq. Is it the future that raises an IOE, or the operation? I can see both needing to

Yes, both need to raises some kind of Exception.  The async methods may or may not throw IOException.
 We list a few possible choices below.
# The methods do not throw IOException.  They only submit a task for execution and immediately
return a Future object f.  The IOException, if there is any, will be thrown wrapped as an
ExecutionException when calling f.get().  In this case, we need separated threads (or a thread
pool) to submit the RPC request.
# The methods submit an RPC request before returning.  An IOException can be thrown by the
methods if there is a the network IO issues.  The RemoteException and later IOException, if
there is any, will be thrown wrapped as an ExecutionException as in (1).  We do not need separated
threads in this case.
# The methods perform some sanity check (e.g. checkOpen()) before returning.  An IOException
can be thrown by the methods if the check fails.  The IOException after the check will be
thrown wrapped as an ExecutionException as in (1).  Similar to (1), we need separated threads
to send the RPC.

(*) We probably should declare the methods with “throw IOException” initially.  We may
remove it later if we strongly believe that “throw IOException” is useless.  If the methods
are declared without “throw IOException”, we may not be able to add it later since it
is an incompatible change.
(*)(*) We may let the user passing an Executor object when initialing FutureFileSystem so
that if it is null, (2) will be used.  Otherwise, (1) or (3) will be used.

bq. assuming this is targed @ Hadoop 3, it'd be nice to make sure this works really well with
the Java 8 language features; maybe this could be the first use in the codebase.
Supporting Java 8 language features is outside the scope of this JIRA.  If we want to support
Java 8 language features, we should also support it for FileSystem.


> Add new FileSystem API to support asynchronous method calls
> -----------------------------------------------------------
>
>                 Key: HADOOP-12910
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12910
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Xiaobing Zhou
>
> Add a new API, namely FutureFileSystem (or AsynchronousFileSystem, if it is a better
name).  All the APIs in FutureFileSystem are the same as FileSystem except that the return
type is wrapped by Future, e.g.
> {code}
>   //FileSystem
>   public boolean rename(Path src, Path dst) throws IOException;
>   //FutureFileSystem
>   public Future<Boolean> rename(Path src, Path dst) throws IOException;
> {code}
> Note that FutureFileSystem does not extend FileSystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message