beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tg...@apache.org
Subject [1/2] beam git commit: Throw an Exception if no files are found to stage
Date Mon, 14 Aug 2017 21:28:12 GMT
Repository: beam
Updated Branches:
  refs/heads/master b0b642182 -> 0f8e8dd70


Throw an Exception if no files are found to stage

We should always stage the user's JAR. If we don't find any files and
none were specified, then the pipeline will not execute, and this should
fail early rather than later.


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e73ae399
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e73ae399
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e73ae399

Branch: refs/heads/master
Commit: e73ae399fcda37ad33dd0e1c04cf0eb3c0548473
Parents: b0b6421
Author: Kamil Szewczyk <szewinho@gmail.com>
Authored: Mon Jul 17 13:54:10 2017 +0200
Committer: Thomas Groh <tgroh@google.com>
Committed: Mon Aug 14 13:46:32 2017 -0700

----------------------------------------------------------------------
 .../apache/beam/runners/dataflow/DataflowRunner.java  | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/beam/blob/e73ae399/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
----------------------------------------------------------------------
diff --git a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
index 6999616..496681e 100644
--- a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
+++ b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
@@ -243,11 +243,15 @@ public class DataflowRunner extends PipelineRunner<DataflowPipelineJob>
{
     if (dataflowOptions.getFilesToStage() == null) {
       dataflowOptions.setFilesToStage(detectClassPathResourcesToStage(
           DataflowRunner.class.getClassLoader()));
-      LOG.info("PipelineOptions.filesToStage was not specified. "
-          + "Defaulting to files from the classpath: will stage {} files. "
-          + "Enable logging at DEBUG level to see which files will be staged.",
-          dataflowOptions.getFilesToStage().size());
-      LOG.debug("Classpath elements: {}", dataflowOptions.getFilesToStage());
+      if (dataflowOptions.getFilesToStage().isEmpty()) {
+        throw new IllegalArgumentException("No files to stage has been found.");
+      } else {
+        LOG.info("PipelineOptions.filesToStage was not specified. "
+                        + "Defaulting to files from the classpath: will stage {} files. "
+                        + "Enable logging at DEBUG level to see which files will be staged.",
+                dataflowOptions.getFilesToStage().size());
+        LOG.debug("Classpath elements: {}", dataflowOptions.getFilesToStage());
+      }
     }
 
     // Verify jobName according to service requirements, truncating converting to lowercase
if


Mime
View raw message