ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Lantukh <ilant...@gridgain.com>
Subject Re: Small value for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE
Date Mon, 03 Jul 2017 13:18:21 GMT
Yes, such races during pre-loading are possible even if you configured 0
backups.

On Fri, Jun 30, 2017 at 7:31 PM, charly <cdessart@appnexus.com> wrote:

> It helps a lot. Thanks for the detailed example. I have a last question.
> Would it still be a potential issue to lower the queue size if we don't
> have any backup? Like if a new node pre-loads from the primary, any new
> changes will also be populated to the soon-to-be new primary?
>
> Sent from my iPhone
>
> On Jun 30, 2017, at 11:27 AM, Ilya Lantukh [via Apache Ignite Users] <[hidden
> email] <http:///user/SendEmail.jtp?type=node&node=14183&i=0>> wrote:
>
> Hi Charly,
>
> There is no special thread in charge of clearing the queue. Entry is
> cleared when you try to add new entry to the queue, but it is already full.
> So, once queue becomes full, it will always stay full, and lowering queue
> size obviously reduces heap consumption. If cache is destroyed, the queue
> should be cleared immediately.
>
> However, if queue size is too low, it may lead to data inconsistency on
> unstable topology.
>
> Here is an example:
> Node A just joined the cluster, became primary node for some partitions
> and started preloading from node B, which is backup. Node B sends value V
> for key K, but before it was received and processed on node A user executed
> cache.remove(K).
> - If queue size is 0, node A now doesn't have any information that entry
> was just removed and has to store (K, V) pair from node B. On the other
> hand, node B will remove K and now primary and backup nodes have different
> data.
> - If queue size is N > 0, node A will save a tombstone entry for K. It
> will have all necessary information to understand that (K, V) pair from
> node B is older than current value and don't store it. Tombstone will be
> cleared after N other keys are removed.
>
> Hope this helps.
>
>
> On Thu, Jun 22, 2017 at 2:11 AM, charly <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=14182&i=0>> wrote:
>
>> Hey everyone,
>>
>> We use Ignite 1.8 and see a difference in heap used when lowering the
>> value
>> for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE. I'm not sure to understand
>> what
>> it's going to happen if we decrease that value on production. Could
>> someone
>> clear that up for me please? From what I understand, deleted entries go in
>> that queue for cleaning, where one thread per cache is in charge of
>> cleaning
>> the queue. But what if the queue is full? I might have missed that
>> information in the documentation, my bad if I did.
>>
>> Maybe on the same subject, is it going to slow down the free up of the
>> heap
>> after cache.destroy()? We have a use-case where we delete 4 caches at the
>> same time containing ~500k entries heavily indexed. When doing so, it
>> takes
>> few minutes for the heap to decrease as you can see here
>> http://imgur.com/a/kPsSc . We would not want to extend the time to free
>> up
>> memory even more.
>>
>> More information about our setup:
>>  - on heap (we'll eventually move to off heap but back then we could not
>> make Ignite 1.8 to free up memory at all with our version of glibc. It
>> would
>> work with jmalloc but that was not permitted on production at that time).
>>  - no backup
>>  - atomicity mode: atomic
>>  - cache mode: partitioned
>>  - cluster of 4 nodes
>>  - jvm options: -J-DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=300000 -J-Xms12g
>> -J-Xmx12g -J-server -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC
>> -J-XX:+UseTLAB -J-XX:NewSize=128m -J-XX:MaxNewSize=128m
>> -J-XX:MaxTenuringThreshold=0 -J-XX:SurvivorRatio=1024
>> -J-XX:+UseCMSInitiatingOccupancyOnly -J-XX:CMSInitiatingOccupancyFr
>> action=10
>> -J-XX:MaxGCPauseMillis=1000 -J-XX:InitiatingHeapOccupancyPercent=10
>> -J-XX:+UseCompressedOops -J-XX:ParallelGCThreads=8 -J-XX:ConcGCThreads=8
>> -J-XX:+PrintGCDetails -J-XX:+PrintGCTimeStamps -J-XX:+PrintGCDateStamps
>> -J-XX:+UseGCLogFileRotation -J-XX:NumberOfGCLogFiles=10
>> -J-XX:GCLogFileSize=100M -J-Xloggc:/tmp/ignite-gc.log
>> -J-Dcom.sun.management.jmxremote.port=49130
>>
>> Thanks for your help,
>> Charly
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE
>> -HISTORY-SIZE-tp14037.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Best regards,
> Ilya
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Small-
> value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037p14182.html
> To unsubscribe from Small value for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE,
> click here.
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
> ------------------------------
> View this message in context: Re: Small value for
> IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE
> <http://apache-ignite-users.70518.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037p14183.html>
>
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>



-- 
Best regards,
Ilya

Mime
View raw message