beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rober...@apache.org
Subject [1/3] beam git commit: Ensure transforms are picklable before materializing to protos.
Date Sat, 01 Apr 2017 04:44:52 GMT
Repository: beam
Updated Branches:
  refs/heads/master 07499824b -> 60901f876


Ensure transforms are picklable before materializing to protos.


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bd40c5a2
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/bd40c5a2
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/bd40c5a2

Branch: refs/heads/master
Commit: bd40c5a27e379914914a30f79854fd7a38621c66
Parents: c9ff44a
Author: Robert Bradshaw <robertwb@gmail.com>
Authored: Fri Mar 31 16:57:22 2017 -0700
Committer: Robert Bradshaw <robertwb@gmail.com>
Committed: Fri Mar 31 21:44:21 2017 -0700

----------------------------------------------------------------------
 sdks/python/apache_beam/pipeline.py | 6 ++++++
 1 file changed, 6 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/beam/blob/bd40c5a2/sdks/python/apache_beam/pipeline.py
----------------------------------------------------------------------
diff --git a/sdks/python/apache_beam/pipeline.py b/sdks/python/apache_beam/pipeline.py
index 0841e5f..3c416eb 100644
--- a/sdks/python/apache_beam/pipeline.py
+++ b/sdks/python/apache_beam/pipeline.py
@@ -311,6 +311,12 @@ class Pipeline(object):
 
       def visit_transform(self, transform_node):
         if transform_node.side_inputs:
+          # No side inputs (yet).
+          Visitor.ok = False
+        try:
+          # Transforms must be picklable.
+          pickler.loads(pickler.dumps(transform_node.transform))
+        except Exception:
           Visitor.ok = False
     self.visit(Visitor())
     return Visitor.ok


Mime
View raw message