directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raffaele P. Guidi" <raffaele.p.gu...@gmail.com>
Subject Re: Additional Serializer and raw Buffer access
Date Sat, 29 Sep 2012 14:00:27 GMT
What do you think about:
1) implementing a lightning serialization module
2) creating a serializer that directly works on a directmemory provider
ByteBuffer or (maybe better) Unsafe based Pointer?

Now I see lightning is apache licensed and this is fine but I don't think
it is published in any public maven repo, am I right? We could find a way
to deal with this; options vary from publishing lightning to the free
sonatype repo,  joining the ASF (which is great anyhow!) and republishing
lightning code in DirectMemory becoming a commiter (which has to undergo a
PMC vote).

I'd like to hear your and the team feelings on this.

Thanks,
    Raffaele

On Sat, Sep 29, 2012 at 3:27 PM, Noctarius <me@noctarius.com> wrote:

> Hey Raffaele,
>
> that's quite similar to what I did at work. We're developing Flash
> online games and using a customized AMF serialization. To support
> async writing of the clients event results I added serialization of
> the components / entities to the players zone calculation and stored
> the pre-serialized bytestream directly to the off-heap (using
> direct-ring-cache implementation). When the client requests the
> results (using long-polling) I just write the pre-serialized data to
> the right position to deserialize it by standard ways on Flash side.
>
> So yeah an seriliaztion to off-heap algorithm would be fine. You can
> avoid using byte arrays and minimalize GC.
>
> Cheers
> Chris
>
> Am 29.09.2012 15:02, schrieb Raffaele P. Guidi:
> > Nice to hear back from you! Yes, I was thinking about creating a new
> memory
> > storage implementation using Unsafe (and I did it, recently) and also, as
> > DirectMemory relies heavily on serialization (and supports many of them,
> > protostuff, protobuf, msgpack and of course standard serialization),
> about
> > creating a simple embedded serializer leveraging the same techniques you
> > used (Unsafe and bytecode generation).
> > The idea with embedding is avoiding to serialize in a byte array and then
> > moving the byte array to off-heap memory (via Unsafe or ByteBuffers), and
> > serializing directly into a "managed" off-heap buffer, thus further
> > optimizing heap utilization (less GC).
> >
> > What do you think about it? Does it make any sense to you?
> >
> > Ciao,
> >     R
> >
> > On Sat, Sep 29, 2012 at 2:40 PM, Noctarius <me@noctarius.com> wrote:
> >
> >> Hey guys,
> >>
> >> Raffaele found out about a project of mine (Lightning) a few weeks
> >> ago. Lightning is a heavy Unsafe and Bytecode generation using
> >> Serializer implementation. He told me that he was interested in
> >> adding something similar to DirectMemory and I would be glad to help
> >> out in this.
> >>
> >> Another project I started a few days ago, since it was needed for
> >> work is DirectRingCache. The name not really meets to actual
> >> implementation since it's not yet a ring buffer using cache. I used
> >> this for a pre-serialization simple bytestream cache with
> >> self-growing buffers. It could be nice to have DirectMemory having
> >> raw "buffers" to write to or to read from.
> >>
> >> Here are the links from the projects:
> >> https://github.com/noctarius/Lightning
> >> https://github.com/noctarius/direct-ring-cache
> >>
> >> It would be nice to help out since I really like the idea of
> >> DirectMemory and since direct-ring-cache is some kind of reinventing
> >> the wheel.
> >>
> >> Cheers
> >> Noctarius (Chris)
> >>
> >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message