hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15426) S3guard throttle events => 400 error code => exception
Date Tue, 24 Jul 2018 22:27:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554863#comment-16554863
] 

Steve Loughran commented on HADOOP-15426:
-----------------------------------------

Screenshot of the state of play; this from an mvn integration test with parallelism of 6.
Autoscale kicks in, but not enough to stop throttle events coming back. Repeatable.

Happens against US-west from  a laptop in our sunnyvale office; 1+Gbps link; traceroute to
AWS S3 says 20 hobs but a latency of < 3 millis
{code}
20  s3-us-west-1-w.amazonaws.com (54.231.237.43)  2.319 ms  2.175 ms  2.140 ms
{code}

The reason I'm seeing this now is probably that my test system is so close to the DDB and
S3 stores that its overloading, whereas testing from the UK for AWS ireland we've got a tangible
RTT

> S3guard throttle events => 400 error code => exception
> ------------------------------------------------------
>
>                 Key: HADOOP-15426
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15426
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Priority: Major
>         Attachments: Screen Shot 2018-07-24 at 15.16.46.png
>
>
> managed to create on a parallel test run
> {code}
> org.apache.hadoop.fs.s3a.AWSServiceThrottledException: delete on s3a://hwdev-steve-ireland-new/fork-0005/test/existing-dir/existing-file:
com.amazonaws.services.dynamodbv2.model.ProvisionedThroughputExceededException: The level
of configured provisioned throughput for the table was exceeded. Consider increasing your
provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400;
Error Code: ProvisionedThroughputExceededException; Request ID: RDM3370REDBBJQ0SLCLOFC8G43VV4KQNSO5AEMVJF66Q9ASUAAJG):
The level of configured provisioned throughput for the table was exceeded. Consider increasing
your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code:
400; Error Code: ProvisionedThroughputExceededException; Request ID: RDM3370REDBBJQ0SLCLOFC8G43VV4KQNSO5AEMVJF66Q9ASUAAJG)
> 	at 
> {code}
> We should be able to handle this. 400 "bad things happened" error though, not the 503
from S3.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message