beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin BENOIST (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-3224) Add support for path with braces for Google Cloud Storage
Date Mon, 20 Nov 2017 10:46:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benjamin BENOIST updated BEAM-3224:
-----------------------------------
    Description: 
At the moment we can not use braces in Google Cloud Storage paths, as explained [here|https://stackoverflow.com/questions/46977552/filebasedsource-not-able-to-understand-a-glob-corresponding-to-several-specific].

The path is backed by a file pattern defined as a Java glob and is then then expanded to a
regex in _sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/util/GcsUtil.java_
in the _wildcardToRegexp_ function.

{{gs://bucket/{file1,file2,file3}. }} should match {{gs://bucket/file1}}, {{gs://bucket/file2}}
and {{gs://bucket/file3}}

  was:
At the moment we can not use braces in Google Cloud Storage paths, as explained [here|https://stackoverflow.com/questions/46977552/filebasedsource-not-able-to-understand-a-glob-corresponding-to-several-specific].

The path is backed by a file pattern defined as a Java glob and is then then expanded to a
regex in _sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/util/GcsUtil.java_
in the _wildcardToRegexp_ function.

{{gs://bucket/{file1,file2,file3} }} should match {{gs://bucket/file1}}, {{gs://bucket/file2}}
and {{gs://bucket/file3}}


> Add support for path with braces for Google Cloud Storage
> ---------------------------------------------------------
>
>                 Key: BEAM-3224
>                 URL: https://issues.apache.org/jira/browse/BEAM-3224
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-gcp
>            Reporter: Benjamin BENOIST
>            Assignee: Chamikara Jayalath
>            Priority: Minor
>   Original Estimate: 3h
>  Remaining Estimate: 3h
>
> At the moment we can not use braces in Google Cloud Storage paths, as explained [here|https://stackoverflow.com/questions/46977552/filebasedsource-not-able-to-understand-a-glob-corresponding-to-several-specific].
> The path is backed by a file pattern defined as a Java glob and is then then expanded
to a regex in _sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/util/GcsUtil.java_
in the _wildcardToRegexp_ function.
> {{gs://bucket/{file1,file2,file3}. }} should match {{gs://bucket/file1}}, {{gs://bucket/file2}}
and {{gs://bucket/file3}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message