hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] mehakmeet opened a new pull request #3144: HADOOP-17774. bytesRead FS statistic showing twice the correct value in S3A
Date Fri, 25 Jun 2021 11:59:54 GMT

mehakmeet opened a new pull request #3144:
URL: https://github.com/apache/hadoop/pull/3144


   Test command: ```mvn clean verify -Dparallel-tests -DtestsThreadCount=4 -Dscale```
   Region: ap-south-1
   
   ```
   INFO] Results:
   [INFO] 
   [WARNING] Tests run: 568, Failures: 0, Errors: 0, Skipped: 5
   ```
   ```
   [INFO] Results:
   [INFO] 
   [ERROR] Errors: 
   [ERROR]   ITestS3AMiscOperationCost.testGetContentSummaryRoot:96->AbstractS3ACostTest.verifyMetrics:376->lambda$testGetContentSummaryRoot$1:96->getContentSummary:140
» TestTimedOut
   [ERROR]   ITestS3AMiscOperationCost.testGetContentSummaryRoot:96->AbstractS3ACostTest.verifyMetrics:376->lambda$testGetContentSummaryRoot$1:96->getContentSummary:140
» TestTimedOut
   [INFO] 
   [ERROR] Tests run: 1460, Failures: 0, Errors: 2, Skipped: 462
   ```
   ```
   [ERROR] Errors: 
   [ERROR]   ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRecursiveRootListing:267
» TestTimedOut
   [INFO] 
   [ERROR] Tests run: 151, Failures: 2, Errors: 1, Skipped: 28
   ```
   
   Seeing these errors: 
   ```
   [ERROR] Failures: 
   [ERROR]   ITestS3AContractRootDir.testListEmptyRootDirectory:82->AbstractContractRootDirectoryTest.testListEmptyRootDirectory:196->Assert.fail:89
Deleted file: unexpectedly found s3a://mehakmeet-singh-data/user as  S3AFileStatus{path=s3a://mehakmeet-singh-data/user;
isDirectory=true; modification_time=0; access_time=0; owner=mehakmeet.singh; group=mehakmeet.singh;
permission=rwxrwxrwx; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false}
isEmptyDirectory=FALSE eTag=null versionId=null
   [ERROR]   ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:101->Assert.fail:89
After 20 attempts: listing after rm /* not empty
   final [00] S3AFileStatus{path=s3a://mehakmeet-singh-data/user; isDirectory=true; modification_time=0;
access_time=0; owner=mehakmeet.singh; group=mehakmeet.singh; permission=rwxrwxrwx; isSymlink=false;
hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=null versionId=null
   
   ```
   have seen these errors intermittently, due to some issue with DynamoDB table. 
   @steveloughran suggested ```hadoop org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
 -force s3a://example-bucket/```
   But that fails with an error: 
   ```
   2021-06-25 16:18:43,464 INFO service.AbstractService: Service PurgeS3GuardDynamoTable failed
in state STARTED
   -1: Filesystem has no metadata store: s3a://mehakmeet-singh-data
   	at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic.failure(AbstractS3GuardDynamoDBDiagnostic.java:115)
   	at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic.require(AbstractS3GuardDynamoDBDiagnostic.java:94)
   	at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic.bindStore(AbstractS3GuardDynamoDBDiagnostic.java:157)
   	at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic.bindFromCLI(AbstractS3GuardDynamoDBDiagnostic.java:147)
   	at org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable.serviceStart(PurgeS3GuardDynamoTable.java:123)
   	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
   	at org.apache.hadoop.service.launcher.ServiceLauncher.coreServiceLaunch(ServiceLauncher.java:619)
   	at org.apache.hadoop.service.launcher.ServiceLauncher.launchService(ServiceLauncher.java:494)
   	at org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable.serviceMain(DumpS3GuardDynamoTable.java:517)
   	at org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable.main(PurgeS3GuardDynamoTable.java:205)
   2021-06-25 16:18:43,467 INFO util.ExitUtil: Exiting with status -1: Filesystem has no metadata
store: s3a://mehakmeet-singh-data
   ```
   Would like the reviewers to run the aws test suite once in their setup while reviewing
as well. 
   CC: @steveloughran @mukund-thakur @bogthe 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message