hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HADOOP-15796) ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs caching is disabled
Date Tue, 02 Oct 2018 18:29:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Gabor Bota reassigned HADOOP-15796:

    Assignee: Gabor Bota

> ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs caching
is disabled
> -------------------------------------------------------------------------------------------------
>                 Key: HADOOP-15796
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15796
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.1.1
>            Reporter: Gabor Bota
>            Assignee: Gabor Bota
>            Priority: Major
> Sometimes we get {{java.io.IOException: s3a://TEST-BUCKET: FileSystem is closed!}} when
running tests, because of filesystem caching: an fs instance is closed from a test teardown
while another test is still using it. (That should be fixed in another issue.)
> To avoid test errors introduced by closed fs I've disabled fs caching while testing.
This caused the following issue in lots of test:
> {noformat}
> [ERROR] testPartitionsResolution(org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing)
 Time elapsed: 1.994 s  <<< ERROR!
> java.lang.ClassCastException: org.apache.hadoop.fs.s3a.S3AFileSystem cannot be cast to
> 	at org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase.lookupWrapperFS(StagingTestBase.java:145)
> 	at org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase$JobCommitterTest.setupJob(StagingTestBase.java:294)
> {noformat}
> We should fix the casting during these tests.
> To switch of fs caching during a test (mvn verify) run, add the following to the config:
> {code:java}
>   <property>
>     <name>fs.s3a.impl.disable.cache</name>
>     <value>true</value>
>   </property>
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message