beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2150) Support for recursive wildcards in GcsPath
Date Mon, 08 May 2017 17:35:04 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001163#comment-16001163
] 

Daniel Halperin commented on BEAM-2150:
---------------------------------------

Or rather, maybe we should change our thinking. Rather than thinking of this as adding recursive
glob support, we should think of this as relaxing the regex accepted by the transform.

The key thing is whether {{**}} is a meaningful glob -- if so, it's a filesystem and we should
probably try to match {{gsutil}}. But if we just say these are normal regular expressions,
then {{**}} is actually the same as {{*}} and your changes make sense.

I'm fine with _either_, as long as we document this (say, on GCS FileSystem).

> Support for recursive wildcards in GcsPath
> ------------------------------------------
>
>                 Key: BEAM-2150
>                 URL: https://issues.apache.org/jira/browse/BEAM-2150
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-core, sdk-java-gcp
>            Reporter: Devon Meunier
>            Assignee: Devon Meunier
>            Priority: Minor
>
> When working with heavily nested folder structures in Google Cloud Storage, it's great
to make use of recursive wildcards, which the current API explicitly does not support.
> This code hasn't been touched in 2 years so it's likely that simply no one's gotten around
to it yet.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message