hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vrodio...@carrieriq.com>
Subject RE: I am curious how to find hbase replication lag
Date Mon, 19 Aug 2013 21:00:02 GMT

Just simple canary probe approach

Update cluster1:t1:canary_row with a current time every (say) 1 sec
Read time from cluster2:t1:canary_row every second

Compute the difference

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

From: Demai Ni [nidmgg@gmail.com]
Sent: Monday, August 19, 2013 1:28 PM
To: user@hbase.apache.org
Subject: Re: I am curious how to find hbase replication lag

Ted, thanks for connecting the two discussion. a topic that quite some
folks are looking for solutions.

Alex, as far as my study goes, there is no direct/easy way to get the lag
info that is easily consumable by regular users(i.e. not a hbase expert).
>From user perspecitive, the lag can be either time or quantity. Although
current hbase replication metrics contains a lot of good information, third
party tool/monitor has to be applied (JMX, ganglia, etc.)  On 0.94 level,
the metrics info is at regionsever level, on 95.0 such info also at peer
cluster level. I don't think there is plan for a table-level yet.

However, the use case is valid. For example: I am replicating table t1 from
cluster M1 to cluster S1; how many seconds/minutes the lag is? Well, I
haven't found a good solution yet. :-)


On Mon, Aug 19, 2013 at 1:04 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> This is related: http://search-hadoop.com/m/SrEIT1jtzPF
> Cheers
> On Mon, Aug 19, 2013 at 12:50 PM, gordoslocos <gordoslocos@gmail.com>
> wrote:
> > I believe hbase keeps info in zk that gives you the count of pending
> > operations to be replicated. Check into the rz zookeeper node in the
> hbase
> > replication documentation.
> >
> > http://hbase.apache.org/replication.html
> >
> >
> > On 19/08/2013, at 16:42, Alex Newman <posix4e@gmail.com> wrote:
> >
> > > I have setup HBase replication. I want to know how out of date my
> > replicant
> > > cluster is. How does one monitor that?
> > >
> > > -Alex Newman
> >

Confidentiality Notice:  The information contained in this message, including any attachments
hereto, may be confidential and is intended to be read only by the individual or entity to
whom this message is addressed. If the reader of this message is not the intended recipient
or an agent or designee of the intended recipient, please note that any review, use, disclosure
or distribution of this message or its attachments, in any form, is strictly prohibited. 
If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com
and delete or destroy any copy of this message and its attachments.

View raw message