zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norbert Kalmar <nkal...@cloudera.com.INVALID>
Subject Re: log files not being cleaned up despite purgeInterval
Date Fri, 19 Jul 2019 12:23:11 GMT
I would first check the permission on zkCleanup.sh and the bin folder.
Sounds like zookeeper user has no access to the /zk/bin directory.
That might also explain why it is not getting deleted by the zk instance.

And I'm not sure in this one, but did you try giving the full path to the
txn log files like bash -ex ./zkCleanup.sh /data/version-2 -n 3 as root?
I think this script might be expecting the full path, including the
version-2 directory.

Regards,
Norbert

On Fri, Jul 19, 2019 at 2:00 PM Koen De Groote <koen.degroote@limecraft.com>
wrote:

> Hello Norbert,
>
> I've set up a new environment which then reached at least 4 *.log files
> All snapshots and log files are kept in /data/version-2/(default for the
> image)
>
> I went into the zookeeper container and executed:
>
> bash -ex ./zkCleanup.sh /data -n 3
>
> As root, this changes nothing. There are still 4 *.log files
>
> Changing to the zookeeper user, I get the following output:
>
> Path '/zookeeper-3.4.13/bin' does not exist.
> Usage:
> PurgeTxnLog dataLogDir [snapDir] -n count
> dataLogDir -- path to the txn log directory
> snapDir -- path to the snapshot directory
> count -- the number of old snaps/logs you want to keep, value should be
> greater than or equal to 3
>
> And the 4 *.log files still exist.
> Also printing the usage, indicating, to me at least, that something about
> the input is wrong, even though it is identical to the one used as root,
> which did not result in this output.
>
> No actual error messages seem to be printed or logged anywhere.
>
> Not sure what to do next.
>
>
>
> On Fri, Jul 19, 2019 at 11:01 AM Norbert Kalmar
> <nkalmar@cloudera.com.invalid> wrote:
>
> > Hi Koen,
> >
> > It should do just as you said. You can also set
> autopurge.snapRetainCount,
> > bu default it is set to 3, so if you didn't set anything it is not a
> reason
> > to keep old logs.
> >
> > As a plan B you could use zkCleanup.sh [snapshotDir] -n 3 to delete all
> > except the last 3 log files. You can add this to a cron job.
> >
> > As for why the old log files not getting deleted, could be something
> > related to the docker image, maybe a permission problem? Do you see any
> > errors in the server log?
> >
> > Regards,
> > Norbert
> >
> > On Thu, Jul 18, 2019 at 9:25 PM Koen De Groote <
> > koen.degroote@limecraft.com>
> > wrote:
> >
> > > Greetings,
> > >
> > > Working with Zookeeper version 3.4.13 in the official docker image.
> > >
> > > I was under the impression that the setting "autopurge.purgeInterval=1"
> > > meant that log files would be cleaned up every hour.
> > >
> > > Instead, I now find that months of these files are just sitting in
> their
> > > directory, untouched.
> > >
> > > So perhaps I'm wrong about that, but I'm not sure.
> > >
> > > What I wish to achieve is that these log files stop accumulating and
> keep
> > > only the most recent. Is there a way to achieve this? Or are they
> merely
> > > historical and can they be deleted freely?
> > >
> > > Kind regards,
> > > Koen De Groote
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message