hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)
Date Mon, 30 Jul 2018 22:08:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562582#comment-16562582
] 

Gabor Bota commented on HADOOP-14154:
-------------------------------------

Tested against eu-west-1, and got some issues, but seems unrelated/known:


{noformat}
[ERROR] Failures:
[ERROR]   ITestS3AContractGetFileStatusV1List>AbstractContractGetFileStatusTest.testListLocatedStatusEmptyDirectory:129->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
listLocatedStatus(test dir): directory count in 4 directories and 0 files expected:<1>
but was:<4>
[ERROR]   ITestS3AContractGetFileStatusV1List>AbstractContractGetFileStatusTest.testListStatusFiltering:463->AbstractContractGetFileStatusTest.verifyListStatus:534->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
length of listStatus(s3a://cloudera-dev-gabor-ireland/fork-0007/test, org.apache.hadoop.fs.contract.AbstractContractGetFileStatusTest$AllPathsFilter@4f6f98a3
) expected:<2> but was:<4>
[ERROR] Errors:
[ERROR]   ITestS3AFileContextMainOperations>FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile:840
? UnsupportedOperation
[ERROR]   ITestS3GuardToolDynamoDB>AbstractS3GuardToolTestBase.testDestroyNoBucket:309->AbstractS3GuardToolTestBase.run:110
? IllegalArgument
[ERROR]   ITestS3GuardToolLocal>AbstractS3GuardToolTestBase.testDestroyNoBucket:309->AbstractS3GuardToolTestBase.run:110
? IllegalArgument
{noformat}

> Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-14154
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14154
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Rajesh Balamohan
>            Assignee: Gabor Bota
>            Priority: Minor
>         Attachments: HADOOP-14154-HADOOP-13345.001.patch, HADOOP-14154-HADOOP-13345.002.patch,
HADOOP-14154-spec-001.pdf, HADOOP-14154-spec-002.pdf, HADOOP-14154.001.patch
>
>
> Add support for "authoritative mode" for DynamoDBMetadataStore.
> The missing feature is to persist the bit set in {{DirListingMetadata.isAuthoritative}}. 
> This topic has been super confusing for folks so I will also file a documentation Jira
to explain the design better.
> We may want to also rename the DirListingMetadata.isAuthoritative field to .isFullListing
to eliminate the multiple uses and meanings of the word "authoritative".
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message