hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13899) tune dynamodb client & tests
Date Wed, 14 Dec 2016 00:39:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746771#comment-15746771

Aaron Fabbri commented on HADOOP-13899:

+1, modulo [~liuml07]'s comments on the DDB stuff.

One comment, [~steve_l]  

on a failure to init an implementation, fail, rather than fallback to Null. If there is a
problem, it must be considered fatal. Otherwise, if dynamoDB is
being authoritative, a client may think it is using, but as it isn't: corruption

I wouldn't say "corruption" but "loss of consistency".  Since S3Guard makes changes to the
MetadataStore only after those changes are made successfully in S3, losing your MetadataStore
should, at most, mean a return to stock S3A behavior.

I think failing fast here makes sense anyways.  I was originally thinking of people wanting
to continue with an ERROR in "degraded consistency" mode if DynamoDB or their MetadataStore
was not available.  If that were a concern, we could always add a configuration flag "continue
on metadatastore init failure". 

> tune dynamodb client & tests
> ----------------------------
>                 Key: HADOOP-13899
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13899
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: HADOOP-13345
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-13455-HADOOP-13899-003.patch, HADOOP-13899-HADOOP-13345-001.patch,
HADOOP-13899-HADOOP-13345-002.patch, HADOOP-13899-HADOOP-13345-004.patch
> While setting up clients for testing dynamo DB, make the tweaks to the dynamo store and
the s3guard code for better use downstream. These are the kind of things we need to round
off the code for production use.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message