Hi Dejan,
Check if the autopurge feature solves your problem:
http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html#sc_advancedConfiguration
-Flavio
On Thursday, April 16, 2015 9:17 AM, Dejan Markic <Dejan.Markic@mobik-ics.com>
wrote:
Hello all!
We are running 3 ZK servers in ensemble, and ZK is processing a lot of commands per seconds.
There are probably around 300 nodes created/checked/set/get per second.
Since we have only information about live sessions we handle in ZK, we don't need any data
persistency - eg: we can stop all nodes, clean all transaction logs/snapshots, and start them
up again, without any issues.
Since we have a lot of requests/changes, we have moved dataDir onto ramdisk, so we have no
problems with disk IOPS, etc.
Is there a way, to minimze the usage of snapshots/logs so ramdisk would not get filled up?
It happens that transaction logs/snapshots grow so large, that we run out of space on ramdisk.
We issue >/usr/share/zookeeper/bin/zkCleanup.sh -n 3< every 2 minutes, so this should
cleanup the dataDir quite often. Why is >count number of snapshots/logs to keep< limited
to 3 and not below?
I assume, in my setup, I don't even need snapshots/logs to be stored after they are not actively
needed?
So my basic questions are:
- can I somehow get rid of snapshot/logs sooner, more often ... ?
- when is snapshot created? Can it be created sooner, so it would be smaller?
- Is it possible to get rid of snapshot/logs all together?
Thank you for all your inputs and kind regards,
Dejan Markic
|