hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HADOOP-15796) ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs caching is disabled
Date Fri, 14 Dec 2018 16:03:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gabor Bota resolved HADOOP-15796.
---------------------------------
    Resolution: Cannot Reproduce

Closing this as I was not able to reproduce the issue during the testing for HADOOP-15819

> ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs caching
is disabled
> -------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-15796
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15796
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.1.1
>            Reporter: Gabor Bota
>            Assignee: Gabor Bota
>            Priority: Major
>
> Sometimes we get {{java.io.IOException: s3a://TEST-BUCKET: FileSystem is closed!}} when
running tests, because of filesystem caching: an fs instance is closed from a test teardown
while another test is still using it. (That should be fixed in another issue.)
> To avoid test errors introduced by closed fs I've disabled fs caching while testing.
This caused the following issue in lots of test:
> {noformat}
> [ERROR] testPartitionsResolution(org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing)
 Time elapsed: 1.994 s  <<< ERROR!
> java.lang.ClassCastException: org.apache.hadoop.fs.s3a.S3AFileSystem cannot be cast to
org.apache.hadoop.fs.s3a.MockS3AFileSystem
> 	at org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase.lookupWrapperFS(StagingTestBase.java:145)
> 	at org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase$JobCommitterTest.setupJob(StagingTestBase.java:294)
> {noformat}
> We should fix the casting during these tests.
> To switch of fs caching during a test (mvn verify) run, add the following to the config:
> {code:java}
>   <property>
>     <name>fs.s3a.impl.disable.cache</name>
>     <value>true</value>
>   </property>
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message