hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.
Date Wed, 30 Nov 2016 17:03:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709115#comment-15709115
] 

Aaron Fabbri commented on HADOOP-13449:
---------------------------------------

Finished a run of the S3A integration tests.  I see that fixing the MockS3Client factory is
not as simple as my last comment, as you use it for the DynamoDBMetadataStore unit test. 
We can revisit this here or on HADOOP-13589.

Here are the integration test failures I see when I configure the DynamoDBMetadataStore via
core-site.xml:

{code}
Failed tests: 
  ITestS3AContractDelete>AbstractContractDeleteTest.testDeleteNonEmptyDirNonRecursive:78->Assert.fail:88
non recursive delete should have raised an exception, but completed with exit code true
  ITestS3AContractDelete>AbstractContractDeleteTest.testDeleteNonEmptyDirRecursive:94->AbstractFSContractTestBase.assertDeleted:349->Assert.fail:88
Deleted file: unexpectedly found s3a://fabbri-dev/test/testDeleteNonEmptyDirNonRecursive as
 S3AFileStatus{path=s3a://fabbri-dev/test/testDeleteNonEmptyDirNonRecursive; isDirectory=true;
modification_time=0; access_time=0; owner=fabbri; group=fabbri; permission=rwxrwxrwx; isSymlink=false}
isEmptyDirectory=false
  ITestS3AConfiguration.testUsernameFromUGI:481 owner in S3AFileStatus{path=s3a://fabbri-dev/;
isDirectory=true; modification_time=0; access_time=0; owner=fabbri; group=fabbri; permission=rwxrwxrwx;
isSymlink=false} isEmptyDirectory=false expected:<[alice]> but was:<[fabbri]>
  ITestS3AFileOperationCost.testFakeDirectoryDeletion:254->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
after rename(srcFilePath, destFilePath): directories_created expected:<1> but was:<0>
  ITestS3AFileOperationCost.testCostOfGetFileStatusOnNonEmptyDir:139->Assert.fail:88 FileStatus
says directory isempty: S3AFileStatus{path=s3a://fabbri-dev/test/empty; isDirectory=true;
modification_time=0; access_time=0; owner=fabbri; group=fabbri; permission=rwxrwxrwx; isSymlink=false}
isEmptyDirectory=true
ls s3a://fabbri-dev/test/empty [00] S3AFileStatus{path=s3a://fabbri-dev/test/empty/simple.txt;
isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1480497225005;
access_time=0; owner=fabbri; group=fabbri; permission=rw-rw-rw-; isSymlink=false} isEmptyDirectory=false

Tests in error: 
  ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:116
» PathIO
  ITestS3AFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryAsNonExistentDirectory:1038->FileContextMainOperationsBaseTest.testRenameDirectoryAsNonExistentDirectory:1052->FileContextMainOperationsBaseTest.rename:1197
» IO
  ITestS3AAWSCredentialsProvider.testAnonymousProvider:133 » AWSServiceIO initia...
  ITestS3AAWSCredentialsProvider.testBadCredentials:102->createFailingFS:76 » AWSServiceIO
  ITestS3ACredentialsInURL.testInstantiateFromURL:86 » AWSClientIO initializing ...
  ITestS3AFileSystemContract>FileSystemContractBaseTest.testWriteReadAndDeleteOneBlock:266->FileSystemContractBaseTest.writeReadAndDelete:285->FileSystemContractBaseTest.writeAndRead:815
» FileAlreadyExists
  ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:656->FileSystemContractBaseTest.rename:512
» AWSServiceIO
{code}


> S3Guard: Implement DynamoDBMetadataStore.
> -----------------------------------------
>
>                 Key: HADOOP-13449
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13449
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Chris Nauroth
>            Assignee: Mingliang Liu
>         Attachments: HADOOP-13449-HADOOP-13345.000.patch, HADOOP-13449-HADOOP-13345.001.patch,
HADOOP-13449-HADOOP-13345.002.patch, HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch,
HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch, HADOOP-13449-HADOOP-13345.007.patch,
HADOOP-13449-HADOOP-13345.008.patch, HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch
>
>
> Provide an implementation of the metadata store backed by DynamoDB.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message