flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chesnay Schepler (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-8406) BucketingSink does not detect hadoop file systems
Date Wed, 10 Jan 2018 13:46:00 GMT
Chesnay Schepler created FLINK-8406:

             Summary: BucketingSink does not detect hadoop file systems
                 Key: FLINK-8406
                 URL: https://issues.apache.org/jira/browse/FLINK-8406
             Project: Flink
          Issue Type: Bug
          Components: FileSystem
    Affects Versions: 1.4.0, 1.5.0
            Reporter: Chesnay Schepler
            Priority: Blocker

In {{BucketingSink#createHadoopFileSystem}} one can find this piece of code:


final org.apache.flink.core.fs.FileSystem flinkFs =
final FileSystem hadoopFs = (flinkFs instanceof HadoopFileSystem)
	? ((HadoopFileSystem) flinkFs).getHadoopFileSystem()
	: null;

{{FileSystem#get()}} wraps the created {{FileSystem}} in a {{SafetyNetWrapperFileSystem}},
resulting in the instanceof check to categorically fail.

We may want to replace the {{get()}} call with {{getUnguardedFileSystem()}}. We should also
look for other occurrences of similar instanceof checks.

According to a thread on the mailing list this causes the BucketingSink to be unusable.

This message was sent by Atlassian JIRA

View raw message