kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Stopford (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4179) Replication Throttling: Add Usability Metrics PartitionBytesInRate & SumReplicaLag
Date Fri, 16 Sep 2016 08:10:20 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ben Stopford updated KAFKA-4179:
--------------------------------
    Affects Version/s: 0.10.1.0
          Description: 
Add two new metrics to Kafka 

PartitionBytesInRate: Equivalent to BytesInPerSec, but at a partition level (i.e. total traffic
- throttled and not throttled). This is required for estimating how long a rebalance will
take to complete. B/s. See usability section below.

SumReplicaLag: This is the sum of all replica lag values on the broker. This metric is used
to monitor progress of a rebalance and is particularly useful for determining if the rebalance
has become stuck due to an overly harsh throttle value (as the metric will stop decreasing).

As covered in KIP-73 https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas

These make it possible for an administrator to calculate how long a rebalance will take. 

> Replication Throttling: Add Usability Metrics PartitionBytesInRate & SumReplicaLag
> ----------------------------------------------------------------------------------
>
>                 Key: KAFKA-4179
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4179
>             Project: Kafka
>          Issue Type: Improvement
>          Components: replication
>    Affects Versions: 0.10.1.0
>            Reporter: Ben Stopford
>
> Add two new metrics to Kafka 
> PartitionBytesInRate: Equivalent to BytesInPerSec, but at a partition level (i.e. total
traffic - throttled and not throttled). This is required for estimating how long a rebalance
will take to complete. B/s. See usability section below.
> SumReplicaLag: This is the sum of all replica lag values on the broker. This metric is
used to monitor progress of a rebalance and is particularly useful for determining if the
rebalance has become stuck due to an overly harsh throttle value (as the metric will stop
decreasing).
> As covered in KIP-73 https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas
> These make it possible for an administrator to calculate how long a rebalance will take.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message