beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem
Date Thu, 22 Jun 2017 17:12:00 GMT
Luke Cwik created BEAM-2500:
-------------------------------

             Summary: Add support for S3 as a Apache Beam FileSystem
                 Key: BEAM-2500
                 URL: https://issues.apache.org/jira/browse/BEAM-2500
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-extensions
            Reporter: Luke Cwik
            Assignee: Davor Bonaci
            Priority: Minor


Note that this is for providing direct integration with S3 as an Apache Beam FileSystem.

There is already support for using the Hadoop S3 connector by depending on the Hadoop File
System module[1], configuring HadoopFileSystemOptions[2] with a S3 configuration[3].

1: https://github.com/apache/beam/tree/master/sdks/java/io/hadoop-file-system
2: https://github.com/apache/beam/blob/master/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystemOptions.java#L53
3: https://wiki.apache.org/hadoop/AmazonS3




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message