zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <fpjunque...@yahoo.com.INVALID>
Subject Re: Transaction logs and snapshots
Date Thu, 16 Apr 2015 09:14:52 GMT
Hi Dejan,
Check if the autopurge feature solves your problem:
http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html#sc_advancedConfiguration
 
-Flavio 


     On Thursday, April 16, 2015 9:17 AM, Dejan Markic <Dejan.Markic@mobik-ics.com>
wrote:
   
 

 Hello all!

We are running 3 ZK servers in ensemble, and ZK is processing a lot of commands per seconds.
There are probably around 300 nodes created/checked/set/get per second.
Since we have only information about live sessions we handle in ZK, we don't need any data
persistency - eg: we can stop all nodes, clean all transaction logs/snapshots, and start them
up again, without any issues.
Since we have a lot of requests/changes, we have moved dataDir onto ramdisk, so we have no
problems with disk IOPS, etc.
Is there a way, to minimze the usage of snapshots/logs so ramdisk would not get filled up?
It happens that transaction logs/snapshots grow so large, that we run out of space on ramdisk.
We issue >/usr/share/zookeeper/bin/zkCleanup.sh -n 3< every 2 minutes, so this should
cleanup the dataDir quite often. Why is >count number of snapshots/logs to keep< limited
to 3 and not below?
I assume, in my setup, I don't even need snapshots/logs to be stored after they are not actively
needed? 
So my basic questions are:  
- can I somehow get rid of snapshot/logs sooner, more often ... ?
- when is snapshot created? Can it be created sooner, so it would be smaller? 
- Is it possible to get rid of snapshot/logs all together?

Thank you for all your inputs and kind regards,
Dejan Markic

 
  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message