zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koen De Groote <koen.degro...@limecraft.com>
Subject Re: log files not being cleaned up despite purgeInterval
Date Mon, 22 Jul 2019 08:13:39 GMT
Performing "bash -ex ./zkCleanup.sh /data/version-2 -n 3" as root results
in the creation of another version-2 folder(empty) in the existing
version-2 folder.

As both root and zookeeper user I am able to create files in the
/data/version-2 directory inside the container.

The zookeeper user is indeed not the owner of anything in the zk/bin
folder(/zookeeper-3.4.13/bin). Executing zkCli.sh works, but creating a
file in there doesn't.

Permission level for the folder seems to be 0755 on all files and the
folder itself.

Just ran into what I think is the problem: the relative path to the zoo.cfg
file isn't correct.

I tried running just plain "./zkCleanup.sh" as the zookeeper user from
within the folder and it printed that it could not find the zoo.cfg file,
but the path it printed was basically "current_dir/../expected_cfg_dir",
which is one ".." too little.

Will check if this is due to a setting of mine.














On Fri, Jul 19, 2019 at 2:23 PM Norbert Kalmar <nkalmar@cloudera.com.invalid>
wrote:

> I would first check the permission on zkCleanup.sh and the bin folder.
> Sounds like zookeeper user has no access to the /zk/bin directory.
> That might also explain why it is not getting deleted by the zk instance.
>
> And I'm not sure in this one, but did you try giving the full path to the
> txn log files like bash -ex ./zkCleanup.sh /data/version-2 -n 3 as root?
> I think this script might be expecting the full path, including the
> version-2 directory.
>
> Regards,
> Norbert
>
> On Fri, Jul 19, 2019 at 2:00 PM Koen De Groote <
> koen.degroote@limecraft.com>
> wrote:
>
> > Hello Norbert,
> >
> > I've set up a new environment which then reached at least 4 *.log files
> > All snapshots and log files are kept in /data/version-2/(default for the
> > image)
> >
> > I went into the zookeeper container and executed:
> >
> > bash -ex ./zkCleanup.sh /data -n 3
> >
> > As root, this changes nothing. There are still 4 *.log files
> >
> > Changing to the zookeeper user, I get the following output:
> >
> > Path '/zookeeper-3.4.13/bin' does not exist.
> > Usage:
> > PurgeTxnLog dataLogDir [snapDir] -n count
> > dataLogDir -- path to the txn log directory
> > snapDir -- path to the snapshot directory
> > count -- the number of old snaps/logs you want to keep, value should be
> > greater than or equal to 3
> >
> > And the 4 *.log files still exist.
> > Also printing the usage, indicating, to me at least, that something about
> > the input is wrong, even though it is identical to the one used as root,
> > which did not result in this output.
> >
> > No actual error messages seem to be printed or logged anywhere.
> >
> > Not sure what to do next.
> >
> >
> >
> > On Fri, Jul 19, 2019 at 11:01 AM Norbert Kalmar
> > <nkalmar@cloudera.com.invalid> wrote:
> >
> > > Hi Koen,
> > >
> > > It should do just as you said. You can also set
> > autopurge.snapRetainCount,
> > > bu default it is set to 3, so if you didn't set anything it is not a
> > reason
> > > to keep old logs.
> > >
> > > As a plan B you could use zkCleanup.sh [snapshotDir] -n 3 to delete all
> > > except the last 3 log files. You can add this to a cron job.
> > >
> > > As for why the old log files not getting deleted, could be something
> > > related to the docker image, maybe a permission problem? Do you see any
> > > errors in the server log?
> > >
> > > Regards,
> > > Norbert
> > >
> > > On Thu, Jul 18, 2019 at 9:25 PM Koen De Groote <
> > > koen.degroote@limecraft.com>
> > > wrote:
> > >
> > > > Greetings,
> > > >
> > > > Working with Zookeeper version 3.4.13 in the official docker image.
> > > >
> > > > I was under the impression that the setting
> "autopurge.purgeInterval=1"
> > > > meant that log files would be cleaned up every hour.
> > > >
> > > > Instead, I now find that months of these files are just sitting in
> > their
> > > > directory, untouched.
> > > >
> > > > So perhaps I'm wrong about that, but I'm not sure.
> > > >
> > > > What I wish to achieve is that these log files stop accumulating and
> > keep
> > > > only the most recent. Is there a way to achieve this? Or are they
> > merely
> > > > historical and can they be deleted freely?
> > > >
> > > > Kind regards,
> > > > Koen De Groote
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message