beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (BEAM-852) Validate sources when they are created
Date Fri, 28 Oct 2016 17:27:58 GMT


ASF GitHub Bot commented on BEAM-852:

GitHub user sb2nov opened a pull request:

    [BEAM-852] Add validation to file based sources during create time

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](
    Adding validation to the fileSourceIO so that we can inform the user if the files don't
exist at the create time. There is a flag to bypass verification incase your local system
doesn't have the same access as the runner etc. 
    PTAL @chamikaramj 

You can merge this pull request into a Git repository by running:

    $ git pull BEAM-852-validate_file_source_at_create

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1220
commit 5e866687c3ef8f3e67838e868a3deaeb9555e460
Author: Sourabh Bajaj <>
Date:   2016-10-28T17:18:51Z

    [BEAM-852] Add validation to file based sources during create time


> Validate sources when they are created
> --------------------------------------
>                 Key: BEAM-852
>                 URL:
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-py
>            Reporter: Chamikara Jayalath
>            Assignee: Sourabh Bajaj
> We currently do not validate some sources at creation time. For example text, Avro. Validating
sources early will improve user experience since it will  help catch issues early. For example,
we can fail before submitting a job to a runner.
> It should also be possible to disable validation to support environments where users
do not have access to the input at job submission. Java SDK already follows a similar model.

This message was sent by Atlassian JIRA

View raw message