beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1624) Unable to deserialize Coder in DataflowRunner
Date Tue, 07 Mar 2017 00:04:32 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898417#comment-15898417
] 

ASF GitHub Bot commented on BEAM-1624:
--------------------------------------

GitHub user kennknowles opened a pull request:

    https://github.com/apache/beam/pull/2173

    [BEAM-1624] Add tests for serialization of BigQueryIO.TableRowInfoCoder

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    I have not yet reproduced the issue in BEAM-1624 (my LeaderBoard pipeline seems happy),
but these tests are a good idea anyhow.
    
    R: @tgroh 
    CC: @francesperry 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kennknowles/beam BQIO-Coder

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/2173.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2173
    
----
commit 57886246a08e82a2cff0d6b95816b76a5b912595
Author: Kenneth Knowles <klk@google.com>
Date:   2017-03-07T00:02:54Z

    Add tests for serialization of BigQueryIO.TableRowInfoCoder

----


> Unable to deserialize Coder in DataflowRunner
> ---------------------------------------------
>
>                 Key: BEAM-1624
>                 URL: https://issues.apache.org/jira/browse/BEAM-1624
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>            Reporter: Frances Perry
>            Assignee: Kenneth Knowles
>            Priority: Blocker
>             Fix For: 0.6.0
>
>
> To repro, sync to head and run the LeaderBoard example with the Dataflow runner
> Does not repro in 0.5.
> Caused by: java.lang.RuntimeException: Unable to deserialize Coder: WindowedValue$FullWindowedValueCoder(KvCoder(BigQueryIO$ShardedKeyCoder(StringUtf8Coder),BigQueryIO$TableRowInfoCoder),IntervalWindow$IntervalWindowCoder).
Check that a suitable constructor is defined.  See Coder for details.
> 	at org.apache.beam.sdk.util.SerializableUtils.ensureSerializable(SerializableUtils.java:115)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$StepTranslator.addOutput(DataflowPipelineTranslator.java:655)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$StepTranslator.addOutput(DataflowPipelineTranslator.java:602)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translateOutputs(DataflowPipelineTranslator.java:945)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator.access$1200(DataflowPipelineTranslator.java:111)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$6.translateMultiHelper(DataflowPipelineTranslator.java:836)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$6.translate(DataflowPipelineTranslator.java:826)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$6.translate(DataflowPipelineTranslator.java:823)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:413)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:486)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:481)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:481)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:481)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:481)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:481)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$400(TransformHierarchy.java:231)
> 	at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:206)
> 	at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:321)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:363)
> 	at org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:153)
> 	at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:505)
> 	at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:150)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:210)
> 	at org.apache.beam.examples.complete.game.GameStats.main(GameStats.java:340)
> 	... 6 more
> Caused by: java.lang.RuntimeException: Unable to deserialize class interface org.apache.beam.sdk.coders.Coder
> 	at org.apache.beam.sdk.util.Serializer.deserialize(Serializer.java:102)
> 	at org.apache.beam.sdk.util.SerializableUtils.ensureSerializable(SerializableUtils.java:112)
> 	... 29 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message