ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasad Bhalerao <prasadbhalerao1...@gmail.com>
Subject Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Date Fri, 11 Jan 2019 10:19:03 GMT
> Resending....
>
>
> I am not able reproduce this issue in small reproducer project but this is
> consistently happening in my project. So I debugged this issue and attached
> the screenshot in this mail.
>
>
>
> *NOTE:  *This issue occurs if the statistics are enabled on cache
> configuration level [cacheCfg.setStatisticsEnabled(true)].
>
>
>
> As shown in screenshots there 22 cache metrics in cacheMetrics hashmap.
> All these cache metrics get serialized successfully. But at the time of
> deserialization on client node only first metrics get de-serialized
> successfully but all other metrics till iteration count 13 are
> de-serialized as null value and on iteration 14 “ref” byte value in
> “OptimizedObjectInputStream.readObject0()” method is read as 81 and code
> throws exception.
>
>
>
> I think this where it is going wrong. The object copy at the time of
> serialization and de-serialization should be same but that’s not happening
> in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
> 22 cacheMetrics are being de-serialized successfully.
>
>
>
> *AffinityJob result being serialized on server:*
>
>
>
>
>
> *[image: server.png][image: server.png]*
>
> *AffinityJob result being de-serialized on client:*
>
> [image: client.jpg]
>
>
> Thanks,
> Prasad
>
> On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <ilya.kasnacheev@gmail.com>
> wrote:
>
>> Hello!
>>
>> Do you have a reproducer project to reliably confirm this issue?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <akashshinde@gmail.com>:
>>
>>> Added  dev@ignite.apache.org.
>>>
>>> Should I log Jira for this issue?
>>>
>>> Thanks,
>>> Akash
>>>
>>>
>>>
>>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <akashshinde@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > No both nodes, client and server are running on Ignite 2.7 version. I
>>> am
>>> > starting both server and client from Intellij IDE.
>>> >
>>> > Version printed in Server node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Version in client node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Thanks,
>>> > Akash
>>> >
>>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mikael-aronsson@telia.com>
>>> wrote:
>>> >
>>> >> Hi!
>>> >>
>>> >> Any chance you might have one node running 2.6 or something like that
>>> ?
>>> >>
>>> >> It looks like it get a different object that does not match the one
>>> >> expected in 2.7
>>> >>
>>> >> Mikael
>>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>> >>
>>> >> Before submitting the affinity task ignite first gets the affinity
>>> cached
>>> >> function (AffinityInfo) by submitting the cluster wide task
>>> "AffinityJob".
>>> >> But while in the process of retrieving the output of this AffinityJob,
>>> >> ignite deserializes this output. I am getting exception while
>>> deserailizing
>>> >> this output.
>>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>>> >> CacheMetrics object from input stream on 14th iteration I am getting
>>> >> following exception. Complete stack trace is given in this mail chain.
>>> >>
>>> >> Caused by: java.io.IOException: Unexpected error occurred during
>>> >> unmarshalling of an instance of the class:
>>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> >>
>>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>> >>
>>> >> Is this a bug or am I doing something wrong?
>>> >>
>>> >> Can someone please help?
>>> >>
>>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <akashshinde@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> When execute affinity.partition(key), I am getting following
>>> exception
>>> >>> on Ignite  2.7.
>>> >>>
>>> >>> Stacktrace:
>>> >>>
>>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>>  at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>>  at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>>  at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed
to
>>> >>> unmarshal object with optimized marshaller
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>>  ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>>  ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>>  ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>>  ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods).
If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt
to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired, closing socket connection
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> >>> 0x3673a59f5dd01c7
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt
to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Session establishment complete on server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed
>>> (SyncConnected)
>>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>>  at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>>  at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>>  at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed
to
>>> >>> unmarshal object with optimized marshaller
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>>  ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>>  ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>>  ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>>  ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods).
If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>>  at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>>  at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>>  at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed
to
>>> >>> unmarshal object with optimized marshaller
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>>  ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>>  ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>>  ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>>  ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods).
If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>>  at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>>  ... 41 common frames omitted
>>> >>>
>>> >>> Thanks,
>>> >>> Akash
>>> >>>
>>> >>
>>>
>>

Mime
View raw message