directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Levenstein <romix...@gmail.com>
Subject Re: KryoSerializer
Date Thu, 01 Nov 2012 16:48:54 GMT
Hi,

On Thu, Nov 1, 2012 at 4:45 PM, Tatu Saloranta <tsaloranta@gmail.com> wrote:
> On Thu, Nov 1, 2012 at 8:37 AM, Christoph Engelbert
> <noctarius@apache.org> wrote:
>> Am 01.11.2012 16:35, schrieb Tatu Saloranta:
>>> On Thu, Nov 1, 2012 at 2:53 AM, Christoph Engelbert
>>> <noctarius@apache.org> wrote:
>>>> Am 30.10.2012 18:01, schrieb Tatu Saloranta:
>>>>> You may want to see how serializer/deserializer in
>>>>>
>>>>> https://github.com/eishay/jvm-serializers
>>>>>
>>>>> is done -- Martin wrote it (or at least modified & has maintained
it),
>>>>> so it should be along best practices.
>>>>>
>>>>> -+ Tatu +-
>>>> This is only best practice for non concurrent access. The unittests
>>>> are executed one by one so the implementation using one byte array
>>>> is not a problem but this cannot be threadsafe.
>>>>
>>>> This applies to this one as well. Never use this implementation in
>>>> concurrent environments :-)
>>>> https://github.com/eishay/jvm-serializers/blob/kannan/tpc/src/serializers/Kryo.java
>>> Correct. What I meant was it was right way to do it for the use case
>>> in question. Which granted requires one to understand the use case...
>>> I forgot that this particular test does not do multi-threaded
>>> processing like others do.
>>>
>>> It actually is one of the things for jvm-serializers to consider;
>>> efficient reuse techniques vary a lot between codecs.
>>>
>>> -+ Tatu +-
>>
>> I guess there's still the problem about the kryo instance itself. On
>> googlecode it is marked that the kryo instance itself is not
>> threadsafe. Anyone has some more informations about that? Tatu? :-)
>
> No, I have only tried to use it without success once or twice, outside
> of jvm-serializers.
> My weapon of choice if Smile+Jakcson. :-D
>
> But Kryo author (Nate something?) should know -- I can try to dig
> contact info, point him to this mailing list.
>
> -+ Tatu +-

I'm one of Kryo's contributors.
Kryo instances do keep some context information during serialization
and deserialization. Therefore, the same instance cannot be safely
used at the same time with multiple input or output streams for
performing simultaneous (de) serializations. And using the same
instance with multiple threads can result in even more problems.

Therefore, when I need to use Kryo in multi-threaded environment, I
usually make sure that each thread gets its own instance of Kryo at
least for the duration of the operation. The reduce the cost of Kryo
instances creation I usually use a pool of such instances. Every time
a thread needs to perform a (de) serialization operation, it would get
an instance from the pool, perform the operation and return the
instance back to the pool. Alternatively, one could use ThreadLocal
Kryo instances if applicable.

-Roman

Mime
View raw message