hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-997) Implement S3 retry mechanism for failed block transfers
Date Tue, 20 Feb 2007 20:49:05 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tom White updated HADOOP-997:
-----------------------------

    Attachment: HADOOP-997-v3.patch

Version 3 patch that fixes the "No such file or directory" problem. This was occurring because
the S3 buffer directory wasn't being created if it didn't exist (tests were OK because hadoop.tmp.dir
is different).

I think this can be committed after review.

> Implement S3 retry mechanism for failed block transfers
> -------------------------------------------------------
>
>                 Key: HADOOP-997
>                 URL: https://issues.apache.org/jira/browse/HADOOP-997
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 0.11.0
>            Reporter: Tom White
>         Assigned To: Tom White
>         Attachments: HADOOP-997-v2.patch, HADOOP-997-v3.patch, HADOOP-997.patch
>
>
> HADOOP-882 improves S3FileSystem so that when certain communications problems with S3
occur the operation is retried. However, the retry mechanism cannot handle a block transfer
failure, since blocks may be very large and we don't want to buffer them in memory. This improvement
is to write a wrapper (using java.lang.reflect.Proxy if possible - see discussion in HADOOP-882)
that can retry block transfers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message