zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dejan Markic <Dejan.Mar...@mobik-ics.com>
Subject Transaction logs and snapshots
Date Thu, 16 Apr 2015 08:17:20 GMT
Hello all!

We are running 3 ZK servers in ensemble, and ZK is processing a lot of commands per seconds.
There are probably around 300 nodes created/checked/set/get per second.
Since we have only information about live sessions we handle in ZK, we don't need any data
persistency - eg: we can stop all nodes, clean all transaction logs/snapshots, and start them
up again, without any issues.
Since we have a lot of requests/changes, we have moved dataDir onto ramdisk, so we have no
problems with disk IOPS, etc.
Is there a way, to minimze the usage of snapshots/logs so ramdisk would not get filled up?
It happens that transaction logs/snapshots grow so large, that we run out of space on ramdisk.
We issue >/usr/share/zookeeper/bin/zkCleanup.sh -n 3< every 2 minutes, so this should
cleanup the dataDir quite often. Why is >count number of snapshots/logs to keep< limited
to 3 and not below?
I assume, in my setup, I don't even need snapshots/logs to be stored after they are not actively
needed? 
So my basic questions are:  
- can I somehow get rid of snapshot/logs sooner, more often ... ?
- when is snapshot created? Can it be created sooner, so it would be smaller? 
- Is it possible to get rid of snapshot/logs all together?

Thank you for all your inputs and kind regards,
Dejan Markic

Mime
View raw message