beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-3369) Python HEAD fails tests due to a ValueError
Date Tue, 19 Dec 2017 20:20:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297373#comment-16297373
] 

ASF GitHub Bot commented on BEAM-3369:
--------------------------------------

robertwb closed pull request #4292: [BEAM-3369] Fix typehint errors introduced in PR #4282.
URL: https://github.com/apache/beam/pull/4292
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/coders/typecoders.py b/sdks/python/apache_beam/coders/typecoders.py
index 797aee5815f..d871c31b2ec 100644
--- a/sdks/python/apache_beam/coders/typecoders.py
+++ b/sdks/python/apache_beam/coders/typecoders.py
@@ -155,6 +155,10 @@ def verify_deterministic(self, key_coder, op_name, silent=True):
         if not silent:
           logging.warning(error_msg)
         return coders.DeterministicFastPrimitivesCoder(key_coder, op_name)
+      elif isinstance(key_coder, coders.TupleCoder):
+        return coders.TupleCoder([
+            self.verify_deterministic(coder, op_name, silent)
+            for coder in key_coder.coders()])
       else:
         raise ValueError(error_msg)
     else:
diff --git a/sdks/python/apache_beam/typehints/opcodes.py b/sdks/python/apache_beam/typehints/opcodes.py
index f0f81ff773f..7fae11b63ea 100644
--- a/sdks/python/apache_beam/typehints/opcodes.py
+++ b/sdks/python/apache_beam/typehints/opcodes.py
@@ -155,7 +155,7 @@ def binary_subscr(state, unused_arg):
   if base in (str, six.text_type):
     out = base
   elif (isinstance(index, Const) and isinstance(index.value, int)
-      and isinstance(base, typehints.TupleHint.TupleConstraint)):
+        and isinstance(base, typehints.TupleHint.TupleConstraint)):
     const_index = index.value
     if -len(base.tuple_types) < const_index < len(base.tuple_types):
       out = base.tuple_types[const_index]


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Python HEAD fails tests due to a ValueError 
> --------------------------------------------
>
>                 Key: BEAM-3369
>                 URL: https://issues.apache.org/jira/browse/BEAM-3369
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>    Affects Versions: Not applicable
>            Reporter: Chamikara Jayalath
>            Assignee: Robert Bradshaw
>            Priority: Critical
>
> tfidf_test.py is failing. Full error message is given below.
> ValueError: The key coder "TupleCoder[BytesCoder, FastPrimitivesCoder]" for GroupByKey
operation "GroupByKey" is not deterministic. This may result in incorrect pipeline output.
This can be fixed by adding a type hint to the operation preceding the GroupByKey step, and
for custom key classes, by writing a deterministic custom Coder. Please see the documentation
for more details.
> Seems to be due to following commit (passes without that).
> https://github.com/apache/beam/commit/1acd1ae901eefbcc8249d90e12ca82db0f91e41e#commitcomment-26356503
> Robert, can you take a look ?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message