flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9305) Register flink-s3-fs-hadoop for the s3a:// scheme as well
Date Mon, 07 May 2018 17:25:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466222#comment-16466222

Stephan Ewen commented on FLINK-9305:

I am not 100% sure about this issue.

Flink's shaded s3a currently does not work with the bucketing sink and will never work with
Hadoop Input and Output formats, because it does not look like a Hadoop FS any more.
This would prevent using vanilla s3a in addition.

One could argue, though, that in that particular case, users should only use vanilla s3a in
the first place.

> Register flink-s3-fs-hadoop for the s3a:// scheme as well
> ---------------------------------------------------------
>                 Key: FLINK-9305
>                 URL: https://issues.apache.org/jira/browse/FLINK-9305
>             Project: Flink
>          Issue Type: Improvement
>          Components: filesystem-connector
>    Affects Versions: 1.4.0, 1.5.0, 1.4.1, 1.4.2, 1.6.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Major
> For enhanced user experience, we should also register our Hadoop S3A-based shaded S3
file system implementation for the {{s3a://}} file system scheme, not just {{s3://}}. This
way, the user can easily switch from the manual S3 integration to the shaded one.

This message was sent by Atlassian JIRA

View raw message