Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 9 Jan 2017 19:51:59 +0000 (UTC)
From: "Mingliang Liu (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13032867.1483877963000.688803.1483991519035@Atlassian.JIRA>
In-Reply-To: <JIRA.13032867.1483877963000@Atlassian.JIRA>
References: <JIRA.13032867.1483877963000@Atlassian.JIRA> <JIRA.13032867.1483877963631@arcas>
Subject: [jira] [Comment Edited] (HADOOP-13959) S3guard: replace
 dynamo.describe() call in init with more efficient query
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 09 Jan 2017 19:52:01 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812667#comment-15812667 ] 

Mingliang Liu edited comment on HADOOP-13959 at 1/9/17 7:51 PM:
----------------------------------------------------------------

+1 for the proposal.

Thanks [~stevel@apache.org] for filing this! This is definitely something we have to do for scaling the S3Guard much better. According to our effort in [HADOOP-13908] v4 patch, we may have to test this patch carefully especially when waiting a newly created table for active. [AWS Forum|https://forums.aws.amazon.com/message.jspa?messageID=759966] has an open discussion thread.


was (Author: liuml07):
+1 for the proposal.

Thanks [~stevel@apache.org] for filing this! This is definitely something we have to do for scaling the S3Guard much better. According to our effort in [HADOOP-13908], we may have to test this patch carefully especially when waiting a newly created table for active. [AWS Forum|https://forums.aws.amazon.com/message.jspa?messageID=759966] has an open discussion thread.

> S3guard: replace dynamo.describe() call in init with more efficient query
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-13959
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13959
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Steve Loughran
>            Priority: Minor
>
> HADOOP-13908 adds initialization when a table isn't created, using the {{describe()}} call.
> AWS document this as inefficient, and throttle it. We should be able to get away with a simple table lookup as the probe


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org