hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.
Date Thu, 01 Dec 2016 12:02:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711793#comment-15711793
] 

Steve Loughran commented on HADOOP-13449:
-----------------------------------------

Some of these tests are failing because dynamo db is trying to init, even when anonymously
authenticating with an external read only store.

we need to consider how to support deployment where some object stores are read only + no
dynamo db, and perhaps fallback to no db if auth fails. And also the situation where one store
has an authoritative DB, another none...it'll have to be on a per-object store basis

{code}

testAnonymousProvider(org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider)  Time elapsed:
0.91 sec  <<< ERROR!
org.apache.hadoop.fs.s3a.AWSServiceIOException: initializing  on s3a://landsat-pds/scene_list.gz:
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: Request is missing Authentication
Token (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: MissingAuthenticationTokenException;
Request ID: NS80UK0G6OKHI6IR7KCIV1VRONVV4KQNSO5AEMVJF66Q9ASUAAJG): Request is missing Authentication
Token (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: MissingAuthenticationTokenException;
Request ID: NS80UK0G6OKHI6IR7KCIV1VRONVV4KQNSO5AEMVJF66Q9ASUAAJG)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1529)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1167)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:948)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:661)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:635)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:618)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$300(AmazonHttpClient.java:586)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:573)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:445)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:1722)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1698)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.createTable(AmazonDynamoDBClient.java:743)
	at com.amazonaws.services.dynamodbv2.document.DynamoDB.createTable(DynamoDB.java:96)
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.createTable(DynamoDBMetadataStore.java:413)
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:187)
	at org.apache.hadoop.fs.s3a.s3guard.S3Guard.getMetadataStore(S3Guard.java:85)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:252)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3246)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3295)
	at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3269)
	at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529)
	at org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider.testAnonymousProvider(ITestS3AAWSCredenti
{code}

> S3Guard: Implement DynamoDBMetadataStore.
> -----------------------------------------
>
>                 Key: HADOOP-13449
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13449
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Chris Nauroth
>            Assignee: Mingliang Liu
>         Attachments: HADOOP-13449-HADOOP-13345.000.patch, HADOOP-13449-HADOOP-13345.001.patch,
HADOOP-13449-HADOOP-13345.002.patch, HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch,
HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch, HADOOP-13449-HADOOP-13345.007.patch,
HADOOP-13449-HADOOP-13345.008.patch, HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch
>
>
> Provide an implementation of the metadata store backed by DynamoDB.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message