ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schmitt <peter.schmitt....@gmail.com>
Subject Re: OFFHEAP_TIERED and memory fragmentation
Date Wed, 08 Feb 2017 00:22:14 GMT
Hi Alexey,

I see. That sounds reasonable on that level (and is something different).
What is your plan to limit the overhead/impact of detecting and moving
around such small chunks?

Kind regards,

2017-02-03 14:52 GMT+01:00 Alexey Goncharuk <alexey.goncharuk@gmail.com>:

> Peter,
> A page is a minimal entity that is allowed to be moved on OS level. If I
> have a chunk of memory allocated in the middle of a page, OS won't be able
> to move it during defragmentation (at least this is how things worked last
> time I checked). In the new approach, we are able to move this chunk within
> the page, this is the key difference compared to the old off-heap approach.
> Note, that fine-grained per-entry locking does not imply a byte array per
> entry. We will still have our old GridCacheMapEntry object, but it will be
> created only during a cache read or update, and will be GC-ed immediately
> after cache operation completes. A value itself, on the other hand, will be
> stored off-heap. Your measurements confirm that pure off-heap cache
> outperforms both options for on-heap caches in terms of GC, so I do not see
> any problems here.
> Thanks,
> Alexey
> 2017-01-26 23:23 GMT+03:00 Peter Schmitt <peter.schmitt.jee@gmail.com>:
>> Hi Alexey,
>> a modern OS won't give you a physical address. Instead you get virtual
>> memory which allows the OS to support features like defragmentation.
>> "fine-grained per-entry" sounds like a byte-array for every entry.
>> We tested something like that and the result is again higher GC pauses.
>> In our case a FGC with off-heap caches takes e.g. up to 1s.
>> With the same data and using on-heap caches it takes up to 25s.
>> With a byte-array per entry we get about 4s for a FGC.
>> With some optimizations you might get down a bit, however, in view of the
>> FGC time I fear that such an approach causes a significant impact for
>> applications which need to keep huge caches.
>> Kind regards,
>> Peter
>> 2017-01-26 15:57 GMT+01:00 Alexey Goncharuk <alexey.goncharuk@gmail.com>:
>>> Hi Peter,
>>> Leaving defragmentation to Ignite is one of the reasons we are trying
>>> PageMemory approach. In Ignite 1.x we basically use OS memory allocator to
>>> place a value off-heap. Once the OS has given us a pointer, the memory
>>> cannot be moved around unless we free this region, thus the fragmentation.
>>> On the other hand, with PageMemory approach we can define the page layout
>>> ourselves and, given fine-grained per-entry locking, move values around in
>>> memory. This makes memory management more complicated and usually defines
>>> some restrictions on the data structures, but we expect this will give us
>>> more control over the data.
>>> For example, one of the tricks we can do in the future is cluster-wide
>>> cache snapshots: we simply dump the continuous region of memory on disk and
>>> get a node-local caches snapshot. This is impossible to do using current
>>> off-heap approach.
>>> --AG
>>> 2017-01-25 12:32 GMT+03:00 Peter Schmitt <peter.schmitt.jee@gmail.com>:
>>>> Hi Val,
>>>> fair enough :-)
>>>> However, in the source-code it looks like "Flexible and precise
>>>> per-cache memory limit" is supported already.
>>>> At least each GridUnsafeMemory instance (in each GridCacheContext)
>>>> receives the value from CacheConfiguration.getOffHeapMaxMemory).
>>>> I haven't debugged it, but the instances are created per cache.
>>>> Furthermore, I'm not sure about the memory-page concept, because you
>>>> would get pages on top of pages (used by a modern OS).
>>>> Sounds like even more fragmentation and in this case Ignite would need
>>>> to do the defragmentation.
>>>> Kind regards,
>>>> Peter
>>>> 2017-01-25 1:56 GMT+01:00 vkulichenko <valentin.kulichenko@gmail.com>:
>>>>> I don't think I will be able to explain better than it's done in the
>>>>> ticket
>>>>> :) Current approach is naive and not always effective for multiple
>>>>> reasons.
>>>>> -Val
>>>>> --
>>>>> View this message in context: http://apache-ignite-users.705
>>>>> 18.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp1
>>>>> 0218p10235.html
>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message