zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Fine <af...@apache.org>
Subject Re: Zookeeper general log or “monitor” command
Date Thu, 08 Mar 2018 19:06:30 GMT
You may also want to consider taking a look and reviewing the change proposed here: https://github.com/apache/zookeeper/pull/307

If I understand correctly, a slow operation log with the threshold for logging set to 0 should
accomplish what you are trying to accomplish.

Thanks,
Abe

On Wed, Mar 7, 2018, at 04:30, Andor Molnar wrote:
> Cool, monitoring the transaction makes much sense I think.
> 
> Interesting to see this tool, I didn't know about it yet. For some reason
> it replicates most of the logic in FileSnap and FileTxnLog classes, but at
> the same time it uses ZooKeeper as a dependency.
> 
> Regards,
> Andor
> 
> 
> 
> On Wed, Mar 7, 2018 at 9:33 AM, Nikhil Bafna <zodvik@gmail.com> wrote:
> 
> > As you mentioned, enabling trace/debug might impact performance.
> >
> >
> > Here's the approach I'm using now.
> >
> >    -
> >
> >    Parse the Zookeeper transaction logs. The best repo I've found to do
> >    this is https://github.com/alenca/zklogtool. It's feature packed. The
> >    only downside is that it won't record the read queries.
> >
> >
> > -
> > zodvik
> >
> > On Mon, Mar 5, 2018 at 5:42 PM, Andor Molnar <andor@cloudera.com> wrote:
> >
> > > Hi Zod,
> > >
> > > Have you checked TRACE logging of client requests? (
> > > http://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.
> > html#sc_authOptions
> > > )
> > > Not sure how much different is from what you refer as 'debug logs', but
> > it
> > > might worth a try. It generates log messages like:
> > >
> > > TRACE [CommitProcWorkThread-1:ZooTrace@88][] -
> > > :Esessionid:0x8061f60cfd200000 type:getChildren2 cxid:0x6
> > > zxid:0xfffffffffffffffe txntype:unknown reqpath:/
> > > TRACE [CommitProcWorkThread-1:ZooTrace@88][] -
> > > :Esessionid:0x8061f60cfd200000 type:getData cxid:0x7
> > > zxid:0xfffffffffffffffe txntype:unknown reqpath:/andor2
> > >
> > > Which looks like quite easy to parse and contains all required
> > information.
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > > On Thu, Mar 1, 2018 at 5:33 AM, Zod <zodvik@gmail.com> wrote:
> > >
> > > > Unfortunately, parsing debug log is no-go since it doesn't print the
> > > > complete command.
> > > >
> > > > I've 2 approached in mind as of now
> > > >
> > > > 1. Modify of org.apache.zookeeper.server.LogFormatter to support
> > > > "tailing".
> > > > 2. Use ngrep on my local to monitor the network traffic to remote ZK
> > > host &
> > > > port 2181
> > > >
> > > > ​Will get back with what works better.​
> > > >
> > > > -
> > > > zodvik
> > > >
> > > > On Thu, Mar 1, 2018 at 1:22 AM, Andor Molnar <andor@cloudera.com>
> > wrote:
> > > >
> > > > > Hi Zod,
> > > > >
> > > > > I think there's no such thing in ZooKeeper currently other than debug
> > > > logs.
> > > > > You might want to attach some processing engine to the debug log
feed
> > > and
> > > > > parse and output data which is important for you into MySQL or Redis
> > > > > database.
> > > > >
> > > > > The downside of this approach is debug logging has significant
> > > > performance
> > > > > impact on the entire ensemble, but it might be acceptable for you
> > > > analysis.
> > > > >
> > > > > Regards,
> > > > > Andor
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Feb 27, 2018 at 11:26 PM, Zod <zodvik@gmail.com> wrote:
> > > > >
> > > > > > I want see all commands issued to my Zookeeper cluster, something
> > > like
> > > > > > general log in MySQL or "monitor" command in Redis. How can
this be
> > > > done?
> > > > > >
> > > > > > The intent is to see how Storm uses Zookeeper (for state
> > management &
> > > > > > ack-ing) and a good way to would be to run a sample topology
and
> > see
> > > > all
> > > > > > the commands issued to Zookeeper by Storm.
> > > > > >
> > > > > > I tried enabling debug log for Zookeeper, but that is insufficient
> > > has
> > > > a
> > > > > > lot of noise. For example, issue a create /node prints
> > > > > >
> > > > > > 2018-02-27 18:05:34 ZooKeeperMain [DEBUG] Processing create
> > > > > > -
> > > > > > zodvik
> > > > > >
> > > > >
> > > >
> > >
> >

Mime
View raw message