directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Mehrotra <chetan.mehro...@gmail.com>
Subject Re: CacheService in a concurrent environment
Date Fri, 19 Jul 2013 11:23:38 GMT
Hi Tommaso,

With latest fix the Kryo objects used are unique to per thread as they
are taken from pool. It might be happening that same object which is
being put in cache is also getting modified concurrently.And thus it
throws ConcurrentModificationException while performing serialization
of a collection object.

Also minor observation ...

> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:486)
> org.apache.directmemory.serialization.kryo.KryoSerializer.serialize(KryoSerializer.java:68)

Looking at the current code [1] the line numbers are not matching. May
be some due to some bytecode changes happening.

Chetan Mehrotra
[1] https://github.com/apache/directmemory/blob/trunk/serializers/kryo/src/main/java/org/apache/directmemory/serialization/kryo/KryoSerializer.java#L68

On Fri, Jul 19, 2013 at 1:03 PM, Noctarius <noctarius@apache.org> wrote:
> No I was talking about DM trunk. I'll have a look tomorrow.
>
>
>
> Tommaso Teofili <tommaso.teofili@gmail.com> schrieb:
>
>>I'm using current DM trunk (0.2-SNAPSHOT), or were you meaning Kryo
>>trunk?
>>
>>Regards,
>>Tommaso
>>
>>2013/7/18 Noctarius <noctarius@apache.org>
>>
>>> Hi Tammaso
>>> I guess this is fixed in the current trunk.
>>>
>>> Chris
>>>
>>>
>>>
>>> Tommaso Teofili <tommaso.teofili@gmail.com> schrieb:
>>>
>>> >Hi all,
>>> >
>>> >I'm doing some tests on using Apache DM for HAMA-732 [1][2] where 4
>>> >local
>>> >processes running locally in parallel should use a separate instance
>>of
>>> >a
>>> >CacheService.
>>> >When using Kryo I get this ConcurrentModificationException:
>>> >
>>> >ERROR bsp.BSPTask: Error running bsp setup and bsp function.
>>> >com.esotericsoftware.kryo.KryoException:
>>> >java.util.ConcurrentModificationException
>>> >Serialization trace:
>>> >classes (sun.misc.Launcher$AppClassLoader)
>>> >classLoader (org.apache.hama.HamaConfiguration)
>>> >conf (org.apache.hama.graph.AggregationRunner)
>>> >aggregationRunner (org.apache.hama.graph.GraphJobRunner)
>>> >runner (org.apache.hama.graph.example.PageRank$PageRankVertex)
>>> > at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
>>> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
>>> > at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
>>> >at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
>>> > at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
>>> > at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:549)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:570)
>>> > at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
>>> >at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:549)
>>> > at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:570)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
>>> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:486)
>>> >at
>>>
>>>
>>>org.apache.directmemory.serialization.kryo.KryoSerializer.serialize(KryoSerializer.java:68)
>>> > at
>>>
>>>
>>>org.apache.directmemory.cache.CacheServiceImpl.put(CacheServiceImpl.java:117)
>>> >at
>>>
>>>
>>>org.apache.directmemory.cache.CacheServiceImpl.put(CacheServiceImpl.java:109)
>>> > at
>>>
>>>
>>>org.apache.hama.graph.OffHeapVerticesInfo.addVertex(OffHeapVerticesInfo.java:79)
>>> >at
>>>
>>>org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:412)
>>> > at
>>org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:102)
>>> >at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
>>> > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>>> >at
>>>
>>>org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1260)
>>> >Caused by: java.util.ConcurrentModificationException
>>> >at
>>>
>>>java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
>>> >at java.util.AbstractList$Itr.next(AbstractList.java:343)
>>> > at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:74)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18)
>>> > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
>>> >at
>>>
>>>
>>>com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
>>> > ... 23 more
>>> >
>>> >
>>> >What do you think would be the recommended way of using DM in such a
>>> >setup?
>>> >May instead be a problem with Kryo?
>>> >
>>> >Thanks a lot in advance,
>>> >Tommaso
>>> >
>>> >
>>> >[1] : https://issues.apache.org/jira/browse/HAMA-732
>>> >[2] :
>>> >
>>>
>>https://github.com/tteofili/hama/blob/HAMA-732/graph/src/main/java/org/apache/hama/graph/OffHeapVerticesInfo.java
>>>
>>> --
>>> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail
>>> gesendet.
>
> --
> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

Mime
View raw message