hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13959) S3guard: replace dynamo.describe() call in init with more efficient query
Date Fri, 13 Jan 2017 13:04:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821743#comment-15821743

Steve Loughran commented on HADOOP-13959:

The proposed version marker of HADOOP-13985 can be used here.

# if the request fails with no such entry: bad table.
# if the request fails for wrong  version: fail with message
# if the request fails for low-level reason: pass up

There's one risk here, transient failures during FS launch. This is something which has surfaced
with S3A and bucket existence checks...I've concluded that it's hard to handle elegantly

> S3guard: replace dynamo.describe() call in init with more efficient query
> -------------------------------------------------------------------------
>                 Key: HADOOP-13959
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13959
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Steve Loughran
>            Priority: Minor
> HADOOP-13908 adds initialization when a table isn't created, using the {{describe()}}
> AWS document this as inefficient, and throttle it. We should be able to get away with
a simple table lookup as the probe

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message