hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
Date Tue, 05 Jan 2010 18:14:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796790#action_12796790
] 

Suresh Srinivas commented on HDFS-717:
--------------------------------------

The following are the layers between the application and the Service Implementation (such
as NameNode).
Application <= Client library => <=RPC client=> <= Network => <= RPC
server => <= Service Impl

Key goals:
# InterruptedExceptions in the client library should not be ignored. This will help in clean
application shutdown. InterruptedException on the server side should not be ignored; see below.
# Applications must be able to differentiate between RPC layer exception from the exceptions
in the Service Impl. Applications can choose to retry a request based on different categories
of exceptions received.
# Exceptions declared in the API should be propagated end to end over RPC from the server
to the application. All undeclared exceptions from the Service Impl including InterruptedException
should be handled by the RPC layer.
# Changes needed in applications to move to FileContext from FileSystem should be minimal.

Proposal: 
Exceptions will be organized as shown below.
# IOException
#* exceptions as declared in the RPC API - note the detailed method exception will be declared
even though they are a subclass of IOException
#* RPCException  - exceptions in the rpc layer
#** RPCClientException - exception encountered in RPC client
#** RPCServerException - exception encountered in RPC server
#** UnexpectedServerException - unexpected exception from the Service Impl to RPC handlers.
# RunTimeException
#* HadoopIllegalArgumentException - sublcass of IllegalArgumentException indicates illegal
or inappropriate argument. 
#* HadoopInterruptedException - subclass of RunTimeException thrown on encountering InterruptedException.
#* UnsupportedOperationException - thrown to indicate the requested operation is not supported.

Rationale:
# declared exception should be subclass of IOException as before - no changes here.
# group the rpc exceptions categorized by client side and server side.
# use runtime exception for InterrruptedException - simplifies migration to FileContext. Subclass
of IOException not used as applications might have catch and ignore code.
# HadoopIllegalArgumentException instead of the java IllegalArgumentException - helps differentiate
exception in Hadoop implementation from exception thrown from java libraries. Applications
can choose to catch IllegalArgumentException.
# unsupported operation is indicated by unchecked UnsupportedOperationException - subclass
of IOException not used as applications might have catch and ignore code. Using RunTimeException
since applications cannot recover from this condition.

Implementation details:
InterruptedException handling:
# Client side changes
#* Client library (both API interface and RPC client) and InputStream and OutputStream returned
by FileContext throw unchecked HadoopInterruptedException on InterruptedException.
# Server changes:
#* InterruptedException is currently ignored in the Service Impl layer. With this change the
Service Impl will throw the exception. Methods in protocol classes such as ClientProtocol
will specify InterruptedException in throws clause.
#* On InterruptedException, RPC handlers close the socket connection to the client. Client
handles this failure same as loss of connection.

RPC layer changes
# RPC layer marshalls HadoopInterruptedException, HadoopIllegalArgumentException, UnsupportedException
from Service Impl all the way to the client.
# RPC layer throws RPCClientException, RPCServerException and UnexpectedServerException.

FileContext and AbstractFileSystem and protocol changes:
# Methods in FileContext declare IOException and the relevant subclasses of IOExceptions.
This helps document the specific exceptions thrown and in marshalling the exception from the
server to application over RPC. RPCExceptions are not declared as thrown in FileContext and
AbstractFileSystem, as some implementation might not use RPC layer (Local file system).
example:
{noformat}
public FSDataInputStream open(Path path) throws IOException, FileNotFoundException, AccessDeniedException;
{noformat}
# Protocol methods (such as ClientProtocol) will the throw exceptions similar to FileContext,
along with InterruptedException.

Finally the FileContext will throw the following exceptions. The exception hierarchy is flattened.
The semantics remains as defined in the earlier comments.
# IOException
#* ServerNotReadyException (NameNode safemode etc)
#* OutOfSpaceException for write operations
#* AccessControlException
#* InvalidPathNameException
#* FileNotFoundException
#* FileAlreadyExistsException
#* DirectoryNotEmptyException
#* NotDirectoryException
#* DirectoryNotAllowedException


> Proposal for exceptions thrown by FileContext and Abstract File System
> ----------------------------------------------------------------------
>
>                 Key: HDFS-717
>                 URL: https://issues.apache.org/jira/browse/HDFS-717
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>             Fix For: 0.22.0
>
>
> Currently the APIs in FileContext throw only IOException. Going forward these APIs will
throw more specific exceptions.
> This jira proposes following hierarchy of exceptions to be thrown by FileContext and
AFS (Abstract File System) classes.
> InterruptedException  (java.lang.InterruptedException)
> IOException
>                 /* Following exceptions extend IOException */
>                 FileNotFoundException
>                 FileAlreadyExistsException
>                 DirectoryNotEmptyException
>                 NotDirectoryException
>                 AccessDeniedException
>                 IsDirectoryException
>                 InvalidPathNameException
>                 
>                 FileSystemException
>                                      /* Following exceptions extend FileSystemException
*/
>                                      FileSystemNotReadyException
>                                      ReadOnlyFileSystemException
>                                      QuotaExceededException
>                                      OutOfSpaceException
>                 RemoteException   (java.rmi.RemoteException)
> Most of the IOExceptions above are caused by invalid user input, while FileSystemException
is thrown when FS is in such a state that the requested operation cannot proceed.
> Please note that the proposed RemoteException is from standard java rmi package, which
also extends IOException.
>                             
> HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap
the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException
or FileSystemException.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message