zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jostein Elvaker Haande <jehaa...@gmail.com>
Subject Problem with ZK log files and snapshots
Date Mon, 16 Jul 2018 11:04:00 GMT
Hello,

I'm using ZooKeeper 3.4.12, and I'm facing an issue where the ZK log files
keep up eating disk space. I'm running this on several machines, and the
symptoms I'm seeing are the seem:

- I see a lot of log files
- Rarely, if ever any snapshot files

I started digging into it, and from
https://zookeeper.apache.org/doc/r3.4.12/zookeeperAdmin.html#sc_maintenance
one should use the ZK Java class org.apache.zookeeper.server.PurgeTxnLog
for this. However, seeing as I rarely see any snapshot files, PurgeTxnLog
never performs any cleanup.

Going deeper down, I can see from the documentation that a snapshot is
created every 100K transactions that are kept in the log files, so I was
interested in seeing how man transaction my logs kept. I then made a script
that used the Java class org.apache.zookeeper.server.LogFormatter which
outputs number of transactions on the same line, and I noticed that my ZK
instances saw a very low transaction count. For most of my instances, the
transaction count was below 20K.

Now I know that by setting snapCount to a value lower than the default
100K, it should generate snapshots more frequently, and ZK should then be
able to do this cleanup automatically which is controlled
by autopurge.snapRetainCount and autopurge.purgeInterval.

However, when setting a new value for snapCount, this does not seem to have
any effect on the existing transaction log files. So my question is, how do
I deal with the old transaction logs? How do I proceed to clean this away
safely?

-- 
Yours sincerely Jostein Elvaker Haande
"A free society is a society where it is safe to be unpopular"
- Adlai Stevenson

http://tolecnal.net -- tolecnal at tolecnal dot net

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message