beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chamikara Jayalath (JIRA)" <>
Subject [jira] [Created] (BEAM-3099) Implement HDFS FileSystem for Python SDK
Date Thu, 26 Oct 2017 17:54:00 GMT
Chamikara Jayalath created BEAM-3099:

             Summary: Implement HDFS FileSystem for Python SDK
                 Key: BEAM-3099
             Project: Beam
          Issue Type: New Feature
          Components: sdk-py-core
            Reporter: Chamikara Jayalath
            Assignee: Chamikara Jayalath

Currently Java SDK has HDFS support but Python SDK does not. With current portability efforts
other runners may soon be able to use Python SDK. Having HDFS support will allow these runners
to execute large scale jobs without using GCS. 

Following suggests some libraries that can be used to connect to HDFS from Python.

This message was sent by Atlassian JIRA

View raw message