beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2283) Consider using actual URIs instead of Strings/ResourceIds in relation to FileSystems
Date Fri, 12 May 2017 17:15:04 GMT
Luke Cwik created BEAM-2283:
-------------------------------

             Summary: Consider using actual URIs instead of Strings/ResourceIds in relation
to FileSystems
                 Key: BEAM-2283
                 URL: https://issues.apache.org/jira/browse/BEAM-2283
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-core, sdk-java-extensions, sdk-java-gcp, sdk-py
            Reporter: Luke Cwik


We treat things like URIs because we expect them to have a scheme component and to be able
to resolve a parent/child but fail to treat them as URIs in the internal implementation since
our string versions don't go through URI normalization. This brings up a few issues:
* The cost of implementing and maintaining ResourceIds instead of having users use a standard
URI implementation. This would just require FileSystems to be able to take a string and give
back a URI (to enable them to have custom implementations in case they extend the concept
of URIs with scheme specific extensions).
* The myriad of bugs that will come up because of improper usage of URI like strings and the
assumptions associated with them (like https://issues.apache.org/jira/browse/BEAM-2277)

Note that swapping to URIs adds complexity because:
* Resolving URIs with glob expressions needs to be handled carefully
* FileSystems may need to implement a complicated type instead of ResourceId.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message