hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14488) s34guard localdynamo listStatus fails after renaming file into directory
Date Mon, 05 Jun 2017 14:40:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037047#comment-16037047
] 

Steve Loughran commented on HADOOP-14488:
-----------------------------------------

Stack (with HADOOP-14487 applied)
{code}
2017-06-05 15:33:27,353 [ScalaTest-main-running-S3AConsistencySuite] DEBUG s3a.S3AFileSystem
(S3AFileSystem.java:innerDelete(1551)) - Getting objects for directory prefix cloud-integration/DELAY_LISTING_ME/S3AConsistencySuite/
to delete
- commit *** FAILED ***
  java.lang.IllegalArgumentException: childPath s3a://hwdev-steve-new/cloud-integration/DELAY_LISTING_ME/S3AConsistencySuite/work/task00/part-00
must be a child of s3a://hwdev-steve-new/cloud-integration/DELAY_LISTING_ME/S3AConsistencySuite/work
  at com.google.common.base.Preconditions.checkArgument(Preconditions.java:383)
  at org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata.childStatusToPathKey(DirListingMetadata.java:299)
  at org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata.put(DirListingMetadata.java:221)
  at org.apache.hadoop.fs.s3a.s3guard.S3Guard.dirListingUnion(S3Guard.java:223)
  at org.apache.hadoop.fs.s3a.S3AFileSystem.innerListStatus(S3AFileSystem.java:1688)
  at org.apache.hadoop.fs.s3a.S3AFileSystem.listStatus(S3AFileSystem.java:1640)
  at com.hortonworks.spark.cloud.s3.S3AConsistencySuite$$anonfun$7.apply$mcV$sp(S3AConsistencySuite.scala:82)
  at com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply$mcV$sp(CloudSuiteTrait.scala:62)
  at com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply(CloudSuiteTrait.scala:60)
  at com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply(CloudSuiteTrait.scala:60)
{code}

> s34guard localdynamo listStatus fails after renaming file into directory
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-14488
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14488
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Steve Loughran
>
> Running scala integration test with inconsistent s3 client & local DDB enabled
> {code}
> fs.rename("work/task-00/part-00", work)
> fs.listStatus(work)
> {code}
> The list status work fails with a message about the childStatus not being a child of
the parent. 
> Hypothesis: rename isn't updating the child path entry



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message