Mailing-List: contact commits-help@beam.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@beam.apache.org
Date: Sat, 8 Apr 2017 19:52:41 +0000 (UTC)
From: "Thomas Weise (JIRA)" <jira@apache.org>
To: commits@beam.apache.org
Message-ID: <JIRA.13023764.1480347343000.241970.1491681161574@Atlassian.JIRA>
In-Reply-To: <JIRA.13023764.1480347343000@Atlassian.JIRA>
References: <JIRA.13023764.1480347343000@Atlassian.JIRA> <JIRA.13023764.1480347343154@jira-lw-us.apache.org>
Subject: [jira] [Assigned] (BEAM-1053) ApexGroupByKeyOperator serialization
 issues
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Sat, 08 Apr 2017 19:52:45 -0000


     [ https://issues.apache.org/jira/browse/BEAM-1053?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Weise reassigned BEAM-1053:
----------------------------------

    Assignee: Thomas Weise  (was: Tushar Gosavi)

> ApexGroupByKeyOperator serialization issues
> -------------------------------------------
>
>                 Key: BEAM-1053
>                 URL: https://issues.apache.org/jira/browse/BEAM-1053
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-apex
>    Affects Versions: 0.3.0-incubating
>            Reporter: Sandeep Deshmukh
>            Assignee: Thomas Weise
>
> While trying to use Apex Runner for wordcount program, the ApexGroupByKey=
Operator  fails with following exception.=20
> 2016-11-28 20:54:47,696 INFO com.datatorrent.stram.engine.StreamingContai=
ner: Deploy request: [OperatorDeployInfo[id=3D9,name=3DExtractPayload,type=
=3DGENERIC,checkpoint=3D{583c4b11000000b3, 0, 0},inputs=3D[OperatorDeployIn=
fo.InputDeployInfo[portName=3Dinput,streamId=3Dstream14,sourceNodeId=3D8,so=
urcePortName=3Doutput,locality=3DCONTAINER_LOCAL,partitionMask=3D0,partitio=
nKeys=3D<null>]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName=3D=
output,streamId=3Dstream0,bufferServer=3D<null>]]], OperatorDeployInfo[id=
=3D8,name=3DReadFromHDFS,type=3DINPUT,checkpoint=3D{583c4b11000000b3, 0, 0}=
,inputs=3D[],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName=3Doutp=
ut,streamId=3Dstream14,bufferServer=3D<null>]]], OperatorDeployInfo[id=3D11=
,name=3DApplication.CountWords/Count.PerElement/Init/Map,type=3DGENERIC,che=
ckpoint=3D{583c4b11000000b3, 0, 0},inputs=3D[OperatorDeployInfo.InputDeploy=
Info[portName=3Dinput,streamId=3Dstream17,sourceNodeId=3D10,sourcePortName=
=3Doutput,locality=3DCONTAINER_LOCAL,partitionMask=3D0,partitionKeys=3D<nul=
l>]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName=3Doutput,strea=
mId=3Dstream18,bufferServer=3D<null>]]], OperatorDeployInfo[id=3D10,name=3D=
Application.CountWords/ParDo(ExtractWords),type=3DGENERIC,checkpoint=3D{583=
c4b11000000b3, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[portName=
=3Dinput,streamId=3Dstream0,sourceNodeId=3D9,sourcePortName=3Doutput,locali=
ty=3DCONTAINER_LOCAL,partitionMask=3D0,partitionKeys=3D<null>]],outputs=3D[=
OperatorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream17,b=
ufferServer=3D<null>]]], OperatorDeployInfo[id=3D13,name=3DApplication.Coun=
tWords/Count.PerElement/Count.PerKey/Combine.GroupedValues/AnonymousParDo,t=
ype=3DGENERIC,checkpoint=3D{583c4b11000000b3, 0, 0},inputs=3D[OperatorDeplo=
yInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream12,sourceNodeId=3D1=
2,sourcePortName=3Doutput,locality=3DCONTAINER_LOCAL,partitionMask=3D0,part=
itionKeys=3D<null>]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portNam=
e=3Doutput,streamId=3Dstream3,bufferServer=3D<null>]]], OperatorDeployInfo[=
id=3D14,name=3DWriteToHDFS/Window.Into()/ApexRunner.AssignWindowsAndSetStra=
tegy/AssignWindows/AssignWindows,type=3DGENERIC,checkpoint=3D{583c4b1100000=
0b3, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[portName=3Dinput,st=
reamId=3Dstream3,sourceNodeId=3D13,sourcePortName=3Doutput,locality=3DCONTA=
INER_LOCAL,partitionMask=3D0,partitionKeys=3D<null>]],outputs=3D[OperatorDe=
ployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream10,bufferServe=
r=3Dgoofy]]], OperatorDeployInfo[id=3D12,name=3DApplication.CountWords/Coun=
t.PerElement/Count.PerKey/GroupByKey,type=3DGENERIC,checkpoint=3D{583c4b110=
00000b3, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[portName=3Dinpu=
t,streamId=3Dstream18,sourceNodeId=3D11,sourcePortName=3Doutput,locality=3D=
CONTAINER_LOCAL,partitionMask=3D0,partitionKeys=3D<null>]],outputs=3D[Opera=
torDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream12,buffer=
Server=3D<null>]]]]
> 2016-11-28 20:54:48,922 ERROR com.datatorrent.stram.engine.StreamingConta=
iner: deploy request failed
> com.esotericsoftware.kryo.KryoException: Class cannot be created (missing=
 no-arg constructor): java.nio.HeapByteBuffer
> Serialization trace:
> activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou=
pByKeyOperator)
>         at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.new=
InstantiatorOf(Kryo.java:1228)
>         at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049)
>         at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.create(F=
ieldSerializer.java:547)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(Fie=
ldSerializer.java:523)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:76=
1)
>         at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSe=
rializer.java:135)
>         at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSe=
rializer.java:21)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectF=
ield.java:106)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(Fie=
ldSerializer.java:528)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:76=
1)
>         at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageA=
gent.java:192)
>         at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent=
.java:137)
>         at com.datatorrent.stram.engine.StreamingContainer.deployNodes(St=
reamingContainer.java:915)
>         at com.datatorrent.stram.engine.StreamingContainer.deploy(Streami=
ngContainer.java:863)
>         at com.datatorrent.stram.engine.StreamingContainer.processHeartbe=
atResponse(StreamingContainer.java:821)
>         at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(=
StreamingContainer.java:706)
>         at com.datatorrent.stram.engine.StreamingContainer.main(Streaming=
Container.java:311)
> 2016-11-28 20:54:48,932 ERROR com.datatorrent.stram.engine.StreamingConta=
iner: Fatal exception in container!


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)