directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: New buffer backend
Date Wed, 17 Jul 2013 08:51:40 GMT
2013/7/17 Christoph Engelbert <noctarius@apache.org>

> Am 17.07.2013 09:23, schrieb Tommaso Teofili:
> > Hi Cristoph,
> >
> > I had a second look and it looks ok from my side, just did you ever try
> it
> > within an OSGi environment? I think it'd be worth making sure it works
> > there as we ship bundles.
>
> Hi Tommaso
>
> I use OSGi a lot and most of the code is already classloader aware
> (at least for all the internals) and since the buffer backend does
> not handle serialization and deserialization it should be not a big
> problem to make it eventually working in OSGi (I just had not a test
> yet). The buffer api is already build as a bundle and I just need to
> make minor adjustments - I think.
>

Ok, that's nice to hear, my doubt also comes from the fact that some jars
are not OSGi ready and would need some hack to make the whole thing work (I
am thinking especially to JNA).

If you have time to make such tests than it'd be awesome, just to make sure
we don't affect the whole cache usability (as directmemory-cache depends on
the new backend).

Regards,
Tommaso


>
> Chris
> > Regards,
> > Tommaso
> >
> > 2013/7/15 Tommaso Teofili <tommaso.teofili@gmail.com>
> >
> >> Hi all,
> >>
> >> regarding APIs I'm happy with suggested additions & improvements.
> >> And also the new buffer backend looks nice, I just didn't have time to
> >> have a deeper look at the code but it feels great.
> >> I wonder if this could be used in an OSGi environment or such
> dependencies
> >> from local environment will make OSGi users less happy.
> >> I'll try to have a second look later this week but it looks quite good,
> >> thanks Christoph!
> >>
> >> Regards,
> >> Tommaso
> >>
> >>
> >> 2013/7/13 Christoph Engelbert <noctarius@apache.org>
> >>
> >>> Am 12.07.2013 23:23, schrieb Raffaele P. Guidi:
> >>>> Love it, great job, and expecially the idea of making it usable on its
> >>> own.
> >>>> This is more DirectMemory than DirectMemory ;)
> >>> Thanks Raffaele
> >>>
> >>>  I started it back than as the an external cache for preserialized
> >>> gameupdates (for gameclients) waiting until they're catched using
> >>> the next polling cycle.
> >>>
> >>> Chris
> >>>
> >>>> Ciao,
> >>>>      R
> >>>> Il giorno 12/lug/2013 20:55, "Christoph Engelbert" <
> >>> noctarius@apache.org>
> >>>> ha scritto:
> >>>>
> >>>>> Hey guys
> >>>>>
> >>>>> I finally managed to merge everything together :-)
> >>>>>
> >>>>> As stated a few weeks before I made a partitioned buffer system
for
> >>>>> good performance and less contention.
> >>>>> It had different selection strategies like TLA (Thread Local
> >>>>> Allocation), a simple RoundRobin or (on Linux and Windows) CLA
> >>>>> (Processor Core Local Allocation), whereas the last is done using
OS
> >>>>> calls and JNA.
> >>>>>
> >>>>> It features ByteBuffers for Heap and Offheap as well as Unsafe.
It
> >>>>> has growing buffers (if slice is full a new one is selected) and
can
> >>>>> handle data bigger than Integer.MAX_VALUE (it uses full long
> >>>>> position pointers).
> >>>>>
> >>>>> It is located in directmemory-buffer submodule since it was it's
own
> >>>>> project and it is fully usable even without having to use
> >>>>> DirectMemory (I would suggest to give users the chance to use it
on
> >>>>> their own).
> >>>>>
> >>>>> As stated before it introduces a new dependency and especially a
> >>>>> platform depending one. At least it is a optional dependency and
CLA
> >>>>> is deactivated if JNA is not available on the classpath.
> >>>>>
> >>>>> I although added 3 properties to configure the default strategy
of
> >>>>> creating the PartitionBufferPools:
> >>>>> directmemory.buffer.pooling.disabled: true deactivates pooling and
> >>>>> uses lazy creation and immediate destroying on release
> >>>>> directmemory.buffer.unsafe.enabled: true activates the usage of
> >>>>> sun.misc.Unsafe raw memory access (a check if unsafe is available
is
> >>>>> applied too)
> >>>>> directmemory.buffer.offheap.enabled: true enabled DirectByteBuffer
> >>>>> usage for for the non-unsafe-pools
> >>>>>
> >>>>> I merged it into my local fork of DirectMemory on github [1] but
had
> >>>>> to adjust the API of DirectMemory on some places. I introduced a
> >>>>> MemoryManagerFactory which handles creation of the different
> >>>>> MemoryManagers (as the old ones - partly renamed -
> >>>>> UnsafeMemoryManager and AllocatorMemoryManager) and the new
> >>>>> PartitionBufferMemoryManager.
> >>>>> The Pointer-API is now able to use PartitionBuffers as well as the
> >>>>> old way using byte[].
> >>>>>
> >>>>> I'm not yet finished, working on making all unittests pass again
but
> >>>>> I would appreciate some opinions, discussion on the new API changes.
> >>>>>
> >>>>> Cheers
> >>>>> Chris
> >>>>>
> >>>>> [1] https://github.com/noctarius/directmemory/tree/buffer
> >>>>>
> >>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message