beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From swegner <...@git.apache.org>
Subject [GitHub] incubator-beam pull request #1627: [BEAM-1160] Add option to disable failure...
Date Thu, 15 Dec 2016 18:29:22 GMT
GitHub user swegner opened a pull request:

    https://github.com/apache/incubator-beam/pull/1627

    [BEAM-1160] Add option to disable failures if filePattern resolves to empty

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    Most PTransforms which take a filePattern have construction-time
    validation which checks-- among other things-- that the specified
    filePattern matches at least one file. This is particularly useful
    for catching typos when specifying input files.
    
    Most PTransforms also have an option to disable their
    construction-time validation.  This is generally used when
    validation cannot be performed at construction time: for example
    because the proper credentials aren't available or the input
    specification is late-bound in a template. To allow for these
    scenarios and still guard against typos, FileBasedSource also
    validates that the filePattern matches at least one file at
    runtime.
    
    This change adds the ability FileBasedSource to disable this runtime
    validation, for cases uses case where empty filePatterns should be
    allowed. FileBasedSource gains a new constructor parameter, and
    PTransforms which use FileBasedSource have the option exposed in their
    respective builder APIs.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/swegner/incubator-beam read-allow-empty-glob

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1627.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1627
    
----
commit 43195ba27b4303c852e81fa1de493ec4fec641bb
Author: Scott Wegner <swegner@google.com>
Date:   2016-12-15T18:12:25Z

    Add option to disable failures if filePattern resolves to empty
    
    Most PTransforms which take a filePattern have construction-time
    validation which checks-- among other things-- that the specified
    filePattern matches at least one file. This is particularly useful
    for catching typos when specifying input files.
    
    Most PTransforms also have an option to disable their
    construction-time validation.  This is generally used when
    validation cannot be performed at construction time: for example
    because the proper credentials aren't available or the input
    specification is late-bound in a template. To allow for these
    scenarios and still guard against typos, FileBasedSource also
    validates that the filePattern matches at least one file at
    runtime.
    
    This change adds the ability FileBasedSource to disable this runtime
    validation, for cases uses case where empty filePatterns should be
    allowed. FileBasedSource gains a new constructor parameter, and
    PTransforms which use FileBasedSource have the option exposed in their
    respective builder APIs.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message