beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chamikara Jayalath (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-1167) Jobs that contain FileBasedSources with globs that expand into zero files should fail
Date Thu, 15 Dec 2016 23:54:58 GMT
Chamikara Jayalath created BEAM-1167:
----------------------------------------

             Summary: Jobs that contain FileBasedSources with globs that expand into zero
files should fail
                 Key: BEAM-1167
                 URL: https://issues.apache.org/jira/browse/BEAM-1167
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core, sdk-py
            Reporter: Chamikara Jayalath


Currently we do not fail if a glob that is used to create a FileBasedSource expand into zero
files. This can be error prone since a typo when specifying a glob pattern can result in a
job that silently pass without producing any output. I believe this should be fixed as follows.

If a glob pattern specified as input to a FileBasedSource expand into zero files.
(1) If validation of FileBasedSource is enabled, fail the job at pipeline construction.
(2) If validation is disabled, fail the job at runtime by raising an exception at initial
splitting.

Both Java and Python SDKs have to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message