beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandeep Deshmukh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-1053) ApexGroupByKeyOperator serialization issues
Date Mon, 28 Nov 2016 15:35:58 GMT
Sandeep Deshmukh created BEAM-1053:
--------------------------------------

             Summary: ApexGroupByKeyOperator serialization issues
                 Key: BEAM-1053
                 URL: https://issues.apache.org/jira/browse/BEAM-1053
             Project: Beam
          Issue Type: Bug
          Components: runner-apex
    Affects Versions: 0.3.0-incubating
            Reporter: Sandeep Deshmukh


While trying to use Apex Runner for wordcount program, the ApexGroupByKeyOperator  fails with
following exception. 

2016-11-28 20:54:47,696 INFO com.datatorrent.stram.engine.StreamingContainer: Deploy request:
[OperatorDeployInfo[id=9,name=ExtractPayload,type=GENERIC,checkpoint={583c4b11000000b3, 0,
0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=stream14,sourceNodeId=8,sourcePortName=output,locality=CONTAINER_LOCAL,partitionMask=0,partitionKeys=<null>]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream0,bufferServer=<null>]]],
OperatorDeployInfo[id=8,name=ReadFromHDFS,type=INPUT,checkpoint={583c4b11000000b3, 0, 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream14,bufferServer=<null>]]],
OperatorDeployInfo[id=11,name=Application.CountWords/Count.PerElement/Init/Map,type=GENERIC,checkpoint={583c4b11000000b3,
0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=stream17,sourceNodeId=10,sourcePortName=output,locality=CONTAINER_LOCAL,partitionMask=0,partitionKeys=<null>]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream18,bufferServer=<null>]]],
OperatorDeployInfo[id=10,name=Application.CountWords/ParDo(ExtractWords),type=GENERIC,checkpoint={583c4b11000000b3,
0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=stream0,sourceNodeId=9,sourcePortName=output,locality=CONTAINER_LOCAL,partitionMask=0,partitionKeys=<null>]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream17,bufferServer=<null>]]],
OperatorDeployInfo[id=13,name=Application.CountWords/Count.PerElement/Count.PerKey/Combine.GroupedValues/AnonymousParDo,type=GENERIC,checkpoint={583c4b11000000b3,
0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=stream12,sourceNodeId=12,sourcePortName=output,locality=CONTAINER_LOCAL,partitionMask=0,partitionKeys=<null>]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream3,bufferServer=<null>]]],
OperatorDeployInfo[id=14,name=WriteToHDFS/Window.Into()/ApexRunner.AssignWindowsAndSetStrategy/AssignWindows/AssignWindows,type=GENERIC,checkpoint={583c4b11000000b3,
0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=stream3,sourceNodeId=13,sourcePortName=output,locality=CONTAINER_LOCAL,partitionMask=0,partitionKeys=<null>]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream10,bufferServer=goofy]]],
OperatorDeployInfo[id=12,name=Application.CountWords/Count.PerElement/Count.PerKey/GroupByKey,type=GENERIC,checkpoint={583c4b11000000b3,
0, 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=stream18,sourceNodeId=11,sourcePortName=output,locality=CONTAINER_LOCAL,partitionMask=0,partitionKeys=<null>]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=stream12,bufferServer=<null>]]]]
2016-11-28 20:54:48,922 ERROR com.datatorrent.stram.engine.StreamingContainer: deploy request
failed
com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg constructor):
java.nio.HeapByteBuffer
Serialization trace:
activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGroupByKeyOperator)
        at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstantiatorOf(Kryo.java:1228)
        at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049)
        at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058)
        at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:547)
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:523)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
        at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:135)
        at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
        at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
        at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.java:192)
        at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java:137)
        at com.datatorrent.stram.engine.StreamingContainer.deployNodes(StreamingContainer.java:915)
        at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingContainer.java:863)
        at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:821)
        at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:706)
        at com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:311)
2016-11-28 20:54:48,932 ERROR com.datatorrent.stram.engine.StreamingContainer: Fatal exception
in container!




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message