kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1800) KafkaException was not recorded at the per-topic metrics
Date Thu, 04 Dec 2014 02:02:14 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233813#comment-14233813

Guozhang Wang commented on KAFKA-1800:

Copying [~jjkoshy]'s comment on the RB here:

It would be useful to clarify the comment on why this needed to be moved further up as you
explained offline - i.e., since buffer exhaustion (for example) can happen before the sender
gets a chance to register the metrics.

Also, we should probably discuss on the jira the additional caveat of failed metadata fetches.
i.e., since that happens in the network-client the true record error rate would be higher
than what's counted by sendermetrics.

The options that we have are:
* Expose Sender's maybeRegisterTopicMetrics and use that in NetworkClient maybeUpdateMetadata
if there are no known partitions for a topic
* Keep it as you have it for now and just accept the above discrepancy - (or we could address
that in a separate jira as it is orthogonal).

> KafkaException was not recorded at the per-topic metrics
> --------------------------------------------------------
>                 Key: KAFKA-1800
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1800
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Guozhang Wang
>            Assignee: Guozhang Wang
>             Fix For: 0.9.0
>         Attachments: KAFKA-1800.patch
> When KafkaException was thrown from producer.send() call, it is not recorded on the per-topic
record-error-rate, but only the global error-rate.
> Since users are usually monitoring on the per-topic metrics, loosing all dropped message
counts at this level that are caused by kafka producer thrown exceptions such as BufferExhaustedException
could be very dangerous.

This message was sent by Atlassian JIRA

View raw message