Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8ABB5200CBE for ; Thu, 22 Jun 2017 19:12:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 86E03160BD3; Thu, 22 Jun 2017 17:12:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D1487160BE7 for ; Thu, 22 Jun 2017 19:12:03 +0200 (CEST) Received: (qmail 7896 invoked by uid 500); 22 Jun 2017 17:12:02 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 7882 invoked by uid 99); 22 Jun 2017 17:12:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Jun 2017 17:12:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id BF6FE188ABB for ; Thu, 22 Jun 2017 17:12:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.001 X-Spam-Level: X-Spam-Status: No, score=-100.001 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id uPloaWfNE0FE for ; Thu, 22 Jun 2017 17:12:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id DCAA55F3A1 for ; Thu, 22 Jun 2017 17:12:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6709AE0663 for ; Thu, 22 Jun 2017 17:12:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 203ED21941 for ; Thu, 22 Jun 2017 17:12:00 +0000 (UTC) Date: Thu, 22 Jun 2017 17:12:00 +0000 (UTC) From: "Luke Cwik (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 22 Jun 2017 17:12:04 -0000 Luke Cwik created BEAM-2500: ------------------------------- Summary: Add support for S3 as a Apache Beam FileSystem Key: BEAM-2500 URL: https://issues.apache.org/jira/browse/BEAM-2500 Project: Beam Issue Type: Improvement Components: sdk-java-extensions Reporter: Luke Cwik Assignee: Davor Bonaci Priority: Minor Note that this is for providing direct integration with S3 as an Apache Beam FileSystem. There is already support for using the Hadoop S3 connector by depending on the Hadoop File System module[1], configuring HadoopFileSystemOptions[2] with a S3 configuration[3]. 1: https://github.com/apache/beam/tree/master/sdks/java/io/hadoop-file-system 2: https://github.com/apache/beam/blob/master/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystemOptions.java#L53 3: https://wiki.apache.org/hadoop/AmazonS3 -- This message was sent by Atlassian JIRA (v6.4.14#64029)