directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: CacheService in a concurrent environment
Date Fri, 19 Jul 2013 13:42:32 GMT
I perhaps managed to find the root cause of that, basically I had a cyclic
reference in the serialized object as follows:
X holds Y holds CacheService holds a number of W each holding a reference
to X, the error went away by making X transient (and restoring it back on
deserialization).

Regards,
Tommaso

2013/7/19 Chetan Mehrotra <chetan.mehrotra@gmail.com>

> Hi Tommaso,
>
> With latest fix the Kryo objects used are unique to per thread as they
> are taken from pool. It might be happening that same object which is
> being put in cache is also getting modified concurrently.And thus it
> throws ConcurrentModificationException while performing serialization
> of a collection object.
>
> Also minor observation ...
>
> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:486)
> >
> org.apache.directmemory.serialization.kryo.KryoSerializer.serialize(KryoSerializer.java:68)
>
> Looking at the current code [1] the line numbers are not matching. May
> be some due to some bytecode changes happening.
>
> Chetan Mehrotra
> [1]
> https://github.com/apache/directmemory/blob/trunk/serializers/kryo/src/main/java/org/apache/directmemory/serialization/kryo/KryoSerializer.java#L68
>
> On Fri, Jul 19, 2013 at 1:03 PM, Noctarius <noctarius@apache.org> wrote:
> > No I was talking about DM trunk. I'll have a look tomorrow.
> >
> >
> >
> > Tommaso Teofili <tommaso.teofili@gmail.com> schrieb:
> >
> >>I'm using current DM trunk (0.2-SNAPSHOT), or were you meaning Kryo
> >>trunk?
> >>
> >>Regards,
> >>Tommaso
> >>
> >>2013/7/18 Noctarius <noctarius@apache.org>
> >>
> >>> Hi Tammaso
> >>> I guess this is fixed in the current trunk.
> >>>
> >>> Chris
> >>>
> >>>
> >>>
> >>> Tommaso Teofili <tommaso.teofili@gmail.com> schrieb:
> >>>
> >>> >Hi all,
> >>> >
> >>> >I'm doing some tests on using Apache DM for HAMA-732 [1][2] where 4
> >>> >local
> >>> >processes running locally in parallel should use a separate instance
> >>of
> >>> >a
> >>> >CacheService.
> >>> >When using Kryo I get this ConcurrentModificationException:
> >>> >
> >>> >ERROR bsp.BSPTask: Error running bsp setup and bsp function.
> >>> >com.esotericsoftware.kryo.KryoException:
> >>> >java.util.ConcurrentModificationException
> >>> >Serialization trace:
> >>> >classes (sun.misc.Launcher$AppClassLoader)
> >>> >classLoader (org.apache.hama.HamaConfiguration)
> >>> >conf (org.apache.hama.graph.AggregationRunner)
> >>> >aggregationRunner (org.apache.hama.graph.GraphJobRunner)
> >>> >runner (org.apache.hama.graph.example.PageRank$PageRankVertex)
> >>> > at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
> >>> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
> >>> > at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
> >>> >at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
> >>> > at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
> >>> > at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:549)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:570)
> >>> > at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
> >>> >at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:549)
> >>> > at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:570)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
> >>> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:486)
> >>> >at
> >>>
> >>>
>
> >>>org.apache.directmemory.serialization.kryo.KryoSerializer.serialize(KryoSerializer.java:68)
> >>> > at
> >>>
> >>>
>
> >>>org.apache.directmemory.cache.CacheServiceImpl.put(CacheServiceImpl.java:117)
> >>> >at
> >>>
> >>>
>
> >>>org.apache.directmemory.cache.CacheServiceImpl.put(CacheServiceImpl.java:109)
> >>> > at
> >>>
> >>>
>
> >>>org.apache.hama.graph.OffHeapVerticesInfo.addVertex(OffHeapVerticesInfo.java:79)
> >>> >at
> >>>
>
> >>>org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:412)
> >>> > at
> >>org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:102)
> >>> >at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
> >>> > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
> >>> >at
> >>>
> >>>org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1260)
> >>> >Caused by: java.util.ConcurrentModificationException
> >>> >at
> >>>
> >>>java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
> >>> >at java.util.AbstractList$Itr.next(AbstractList.java:343)
> >>> > at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:74)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18)
> >>> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
> >>> >at
> >>>
> >>>
>
> >>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
> >>> > ... 23 more
> >>> >
> >>> >
> >>> >What do you think would be the recommended way of using DM in such a
> >>> >setup?
> >>> >May instead be a problem with Kryo?
> >>> >
> >>> >Thanks a lot in advance,
> >>> >Tommaso
> >>> >
> >>> >
> >>> >[1] : https://issues.apache.org/jira/browse/HAMA-732
> >>> >[2] :
> >>> >
> >>>
> >>
> https://github.com/tteofili/hama/blob/HAMA-732/graph/src/main/java/org/apache/hama/graph/OffHeapVerticesInfo.java
> >>>
> >>> --
> >>> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail
> >>> gesendet.
> >
> > --
> > Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail
> gesendet.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message