beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré (JIRA) <>
Subject [jira] [Created] (BEAM-138) Extend TextIO to new protocols (and maybe rename to FileIO)
Date Mon, 21 Mar 2016 09:37:25 GMT
Jean-Baptiste Onofré created BEAM-138:

             Summary: Extend TextIO to new protocols (and maybe rename to FileIO)
                 Key: BEAM-138
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-extensions
            Reporter: Jean-Baptiste Onofré
            Assignee: Jean-Baptiste Onofré

The current TextIO supports:
- local file when using directly path like /path/to...
- Google Service file using path like gs:...

On the other hand, we have a contribution (from Tom) to support HDFS.

For an user perspective, it would be easier to use an unique IO supporting different protocol:
- file:
- gs:
- hdfs:
- mvn:
- ...

It would also be convenient to be able to combine protocols and eventually use a different
coder (for instance xml:hdfs:).
In that case, maybe I would make sense to rename TextIO as generic FileIO.

Thoughts ?

This message was sent by Atlassian JIRA

View raw message