ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From charly <cdess...@appnexus.com>
Subject Re: Small value for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE
Date Fri, 30 Jun 2017 16:31:30 GMT
It helps a lot. Thanks for the detailed example. I have a last question. Would it still be
a potential issue to lower the queue size if we don't have any backup? Like if a new node
pre-loads from the primary, any new changes will also be populated to the soon-to-be new primary?

Sent from my iPhone

On Jun 30, 2017, at 11:27 AM, Ilya Lantukh [via Apache Ignite Users] <ml+s70518n14182h83@n6.nabble.com<mailto:ml+s70518n14182h83@n6.nabble.com>>
wrote:

Hi Charly,

There is no special thread in charge of clearing the queue. Entry is cleared when you try
to add new entry to the queue, but it is already full. So, once queue becomes full, it will
always stay full, and lowering queue size obviously reduces heap consumption. If cache is
destroyed, the queue should be cleared immediately.

However, if queue size is too low, it may lead to data inconsistency on unstable topology.

Here is an example:
Node A just joined the cluster, became primary node for some partitions and started preloading
from node B, which is backup. Node B sends value V for key K, but before it was received and
processed on node A user executed cache.remove(K).
- If queue size is 0, node A now doesn't have any information that entry was just removed
and has to store (K, V) pair from node B. On the other hand, node B will remove K and now
primary and backup nodes have different data.
- If queue size is N > 0, node A will save a tombstone entry for K. It will have all necessary
information to understand that (K, V) pair from node B is older than current value and don't
store it. Tombstone will be cleared after N other keys are removed.

Hope this helps.


On Thu, Jun 22, 2017 at 2:11 AM, charly <[hidden email]</user/SendEmail.jtp?type=node&node=14182&i=0>>
wrote:
Hey everyone,

We use Ignite 1.8 and see a difference in heap used when lowering the value
for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE. I'm not sure to understand what
it's going to happen if we decrease that value on production. Could someone
clear that up for me please? From what I understand, deleted entries go in
that queue for cleaning, where one thread per cache is in charge of cleaning
the queue. But what if the queue is full? I might have missed that
information in the documentation, my bad if I did.

Maybe on the same subject, is it going to slow down the free up of the heap
after cache.destroy()? We have a use-case where we delete 4 caches at the
same time containing ~500k entries heavily indexed. When doing so, it takes
few minutes for the heap to decrease as you can see here
http://imgur.com/a/kPsSc . We would not want to extend the time to free up
memory even more.

More information about our setup:
 - on heap (we'll eventually move to off heap but back then we could not
make Ignite 1.8 to free up memory at all with our version of glibc. It would
work with jmalloc but that was not permitted on production at that time).
 - no backup
 - atomicity mode: atomic
 - cache mode: partitioned
 - cluster of 4 nodes
 - jvm options: -J-DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=300000 -J-Xms12g
-J-Xmx12g -J-server -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC
-J-XX:+UseTLAB -J-XX:NewSize=128m -J-XX:MaxNewSize=128m
-J-XX:MaxTenuringThreshold=0 -J-XX:SurvivorRatio=1024
-J-XX:+UseCMSInitiatingOccupancyOnly -J-XX:CMSInitiatingOccupancyFraction=10
-J-XX:MaxGCPauseMillis=1000 -J-XX:InitiatingHeapOccupancyPercent=10
-J-XX:+UseCompressedOops -J-XX:ParallelGCThreads=8 -J-XX:ConcGCThreads=8
-J-XX:+PrintGCDetails -J-XX:+PrintGCTimeStamps -J-XX:+PrintGCDateStamps
-J-XX:+UseGCLogFileRotation -J-XX:NumberOfGCLogFiles=10
-J-XX:GCLogFileSize=100M -J-Xloggc:/tmp/ignite-gc.log
-J-Dcom.sun.management.jmxremote.port=49130

Thanks for your help,
Charly





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com<http://Nabble.com>.



--
Best regards,
Ilya


________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-ignite-users.70518.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037p14182.html
To unsubscribe from Small value for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE, click here<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=14037&code=Y2Rlc3NhcnRAYXBwbmV4dXMuY29tfDE0MDM3fDE3Mzc3NzQ4OTA=>.
NAML<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037p14183.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Mime
View raw message