hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)
Date Mon, 30 Jul 2018 22:05:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562579#comment-16562579

Gabor Bota commented on HADOOP-14154:

Uploaded patch v001 for this issue. 
* I had to fix {{MetadataStoreTestBase#testPruneUnsetsAuthoritative}} because it did not test
anything. (had to add a dir which is authoritative, but a child element gets removed so the
parent dir loses the flag)
* I've added {{testPrunePreservesAuthoritative}} to test that the authoritative flag is preserved
when no child elements pruned from the dir.
* Added {{PathMetadata#isAuthoritativeDir}} because {{DynamoDBMetadataStore}} stores {{PathMetadata}}
in ddb, so that was a logical choice.

Big thanks to [~fabbri] for the continuous help, and for the original idea on how to solve
this issue.

> Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)
> -----------------------------------------------------------------------------
>                 Key: HADOOP-14154
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14154
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Rajesh Balamohan
>            Assignee: Gabor Bota
>            Priority: Minor
>         Attachments: HADOOP-14154-HADOOP-13345.001.patch, HADOOP-14154-HADOOP-13345.002.patch,
HADOOP-14154-spec-001.pdf, HADOOP-14154-spec-002.pdf, HADOOP-14154.001.patch
> Add support for "authoritative mode" for DynamoDBMetadataStore.
> The missing feature is to persist the bit set in {{DirListingMetadata.isAuthoritative}}. 
> This topic has been super confusing for folks so I will also file a documentation Jira
to explain the design better.
> We may want to also rename the DirListingMetadata.isAuthoritative field to .isFullListing
to eliminate the multiple uses and meanings of the word "authoritative".

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message