hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1411) AlreadyBeingCreatedException from task retries
Date Wed, 23 May 2007 01:29:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498073

Hairong Kuang commented on HADOOP-1411:

I did put some logic in the patch to HADOOP-1263 to handle retries for AlreadyBeingCreatedException.
But it turned out it did not work. The problem is that any IPC call returns only RemoteException.
When a server operation throws AlreadyBeingCreatedException, the client only gets RemoteException
and it has to examine the content of RemoteException to find out the real exception. So the
DFSClient retry framework implemented in HADOOP-1263 never catches an AlreadyBeingCreatedException
and therefore it never gets retried.

I'd like to propose the following changes to the general retry framework so it is able to
handle RemoteException well:
Method shouldRetry of RetryPolicies.exceptionDependentRetry checks if the exception is a RemoteException.
If yes, find out the retry policy for the real exception from the exceptionToPolicyMap. Because
RemoteException contains only the class name of the real exception, I would also propose to
change the exceptionToPolicyMap to map an exception class name to a retry policy. Currently
it is a map from a exception class to a retry policy. 

> AlreadyBeingCreatedException from task retries
> ----------------------------------------------
>                 Key: HADOOP-1411
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1411
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.13.0
>            Reporter: Nigel Daley
>         Assigned To: Hairong Kuang
>            Priority: Blocker
>             Fix For: 0.13.0
> HADOOP-1407 indicates 2 bugs: a mapred bug which will be fixed as part of 1407, and a
DFSClient bug that will be fixed here.
> Note that the test run in 1407 was without speculation.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message