ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schmitt <peter.schmitt....@gmail.com>
Subject Re: OFFHEAP_TIERED and memory fragmentation
Date Thu, 26 Jan 2017 20:23:07 GMT
Hi Alexey,

a modern OS won't give you a physical address. Instead you get virtual
memory which allows the OS to support features like defragmentation.
"fine-grained per-entry" sounds like a byte-array for every entry.
We tested something like that and the result is again higher GC pauses.
In our case a FGC with off-heap caches takes e.g. up to 1s.
With the same data and using on-heap caches it takes up to 25s.
With a byte-array per entry we get about 4s for a FGC.
With some optimizations you might get down a bit, however, in view of the
FGC time I fear that such an approach causes a significant impact for
applications which need to keep huge caches.

Kind regards,
Peter



2017-01-26 15:57 GMT+01:00 Alexey Goncharuk <alexey.goncharuk@gmail.com>:

> Hi Peter,
>
> Leaving defragmentation to Ignite is one of the reasons we are trying
> PageMemory approach. In Ignite 1.x we basically use OS memory allocator to
> place a value off-heap. Once the OS has given us a pointer, the memory
> cannot be moved around unless we free this region, thus the fragmentation.
> On the other hand, with PageMemory approach we can define the page layout
> ourselves and, given fine-grained per-entry locking, move values around in
> memory. This makes memory management more complicated and usually defines
> some restrictions on the data structures, but we expect this will give us
> more control over the data.
>
> For example, one of the tricks we can do in the future is cluster-wide
> cache snapshots: we simply dump the continuous region of memory on disk and
> get a node-local caches snapshot. This is impossible to do using current
> off-heap approach.
>
> --AG
>
> 2017-01-25 12:32 GMT+03:00 Peter Schmitt <peter.schmitt.jee@gmail.com>:
>
>> Hi Val,
>>
>> fair enough :-)
>> However, in the source-code it looks like "Flexible and precise
>> per-cache memory limit" is supported already.
>> At least each GridUnsafeMemory instance (in each GridCacheContext)
>> receives the value from CacheConfiguration.getOffHeapMaxMemory).
>> I haven't debugged it, but the instances are created per cache.
>>
>> Furthermore, I'm not sure about the memory-page concept, because you
>> would get pages on top of pages (used by a modern OS).
>> Sounds like even more fragmentation and in this case Ignite would need to
>> do the defragmentation.
>>
>> Kind regards,
>> Peter
>>
>>
>>
>> 2017-01-25 1:56 GMT+01:00 vkulichenko <valentin.kulichenko@gmail.com>:
>>
>>> I don't think I will be able to explain better than it's done in the
>>> ticket
>>> :) Current approach is naive and not always effective for multiple
>>> reasons.
>>>
>>> -Val
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp1
>>> 0218p10235.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>

Mime
View raw message