hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack@archive.org (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-997) Implement S3 retry mechanism for failed block transfers
Date Tue, 13 Feb 2007 19:08:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12472832

stack@archive.org commented on HADOOP-997:

I tried the patch and got this:

hadoop@domU-12-31-33-00-02-D5:/mnt/hadoop-trunk$ ./bin/hadoop fs -fs s3://AWS_ID:AWS_SECRET@BUCKET
-put /mnt/search/searcher.dir/index-sorted /1998/20070207040835-1998
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
        at org.apache.hadoop.fs.s3.$Proxy0.initialize(Unknown Source)
        at org.apache.hadoop.fs.s3.S3FileSystem.initialize(S3FileSystem.java:63)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:160)
        at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:91)
        at org.apache.hadoop.fs.FsShell.init(FsShell.java:39)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:796)
        at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:189)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:895)
Caused by: java.lang.IllegalAccessException: Class org.apache.hadoop.io.retry.RetryInvocationHandler
can not access a member of class org.apache.hadoop.fs.s3.FileSystemStore with modifiers "public
        at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
        at java.lang.reflect.Method.invoke(Method.java:578)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:62)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:42)
        ... 9 more

Before the patch, the above command-line worked.

hadoop@domU-12-31-33-00-02-D5:/mnt/hadoop-trunk$ svn info
Revision: 507114

Other comments on patch, 

+ The retry mechanism looks sweet.
+ Is it true that the Jets3tFileSystemStore#initialize runs after the retry proxy has been
set up so initialization will be retried if an Exception?  Is this intentional?  Perhaps initialization
should be outside the purview of retry.
+ Minor comment: In version 2 of the retry mechanism, you'd be able to narrow retries to happen
only for certain exception types: e.g. for IOExceptions only rather than for Exception as
its currently written.
+ In S3OutputStream#newBackupFile, you replace File createTempFile with your own tmp file
name fabricator.  Any reason why you didn't stay using createTempFile, just use the override
 with signature instead:  http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html#createTempFile(java.lang.String,%20java.lang.String,%20java.io.File)?
(You do this in a few places in this patch).
+ In S3FileSystem.java#createDefaultStore, should the retry params be exposed as configurables?
(I'm fine if the rejoinder is that this makes for too much configuration).
+ On initialize, should Jets3tFileSystemStore clean up its tmp directory?  (If crash, deleteOnExit
won't get a chance to run).
+ If an IOException in retreiveBlock in Jets3tFileSystemStore, you do a closeQuietly on the
out stream.   Its followed by a finally that does closeQuietly on both the in and out stream.
 I don't follow whats going on here.  Why not just leave all closing to the finally block?
 (Suggest adding a comment on why the special case handling of out stream in IOE block).

Good stuff Tom.

> Implement S3 retry mechanism for failed block transfers
> -------------------------------------------------------
>                 Key: HADOOP-997
>                 URL: https://issues.apache.org/jira/browse/HADOOP-997
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 0.11.0
>            Reporter: Tom White
>         Assigned To: Tom White
>         Attachments: HADOOP-997.patch
> HADOOP-882 improves S3FileSystem so that when certain communications problems with S3
occur the operation is retried. However, the retry mechanism cannot handle a block transfer
failure, since blocks may be very large and we don't want to buffer them in memory. This improvement
is to write a wrapper (using java.lang.reflect.Proxy if possible - see discussion in HADOOP-882)
that can retry block transfers.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message