incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Re: Monitoring replication lag/latency in multi DC setup
Date Thu, 06 Sep 2012 04:19:46 GMT
Cassandra exposes lot of metrics through Jconsole. You might be able to get
some information from Jconsole.

On Wed, Sep 5, 2012 at 8:47 PM, Venkat Rama <venkata.s.rama@gmail.com>wrote:

> Thanks for the quick reply, Mohit.    Can we measure/monitor the size of
> Hinted Handoffs?  Would it be a good enough indicator of my back log?
>
> Although we know when a network is flaky, we are interested in knowing how
> much data is piling up in local DC that needs to be transferred.
>
> Greatly appreciate your help.
>
> VR
>
>
> On Wed, Sep 5, 2012 at 8:33 PM, Mohit Anchlia <mohitanchlia@gmail.com>wrote:
>
>> As far as I know Cassandra doesn't use internal queueing mechanism
>> specific to replication. Cassandra sends the write the remote DC and after
>> that it's upto the tcp/ip stack to deal with buffering. If requests starts
>> to timeout Cassandra would use HH upto certain time. For longer outage you
>> would have to run repair.
>>
>> Also look at tcp/ip tuning parameters that are helpful with your scenario:
>>
>> http://kaivanov.blogspot.com/2010/09/linux-tcp-tuning.html
>>
>> Run iperf and test the latency.
>>
>>  On Wed, Sep 5, 2012 at 8:22 PM, Venkat Rama <venkata.s.rama@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> We have multi DC Cassandra ring with 2 DCs setup.   We use LOCAL_QUORUM
>>> for writes and reads.  The network we have seen between the DC is sometimes
>>> flaky lasting few minutes to few 10 of minutes.
>>>
>>> I wanted to know what is the best way to measure/monitor either the lag
>>> or replication latency between the data centers.  Are there any metrics I
>>> can monitor to find the backlog of data that needs to be transferred?
>>>
>>> Thanks in advance.
>>>
>>> VR
>>>
>>
>>
>

Mime
View raw message