kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1800) KafkaException was not recorded at the per-topic metrics
Date Thu, 04 Dec 2014 02:02:14 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233813#comment-14233813
] 

Guozhang Wang commented on KAFKA-1800:
--------------------------------------

Copying [~jjkoshy]'s comment on the RB here:

{code}
It would be useful to clarify the comment on why this needed to be moved further up as you
explained offline - i.e., since buffer exhaustion (for example) can happen before the sender
gets a chance to register the metrics.

Also, we should probably discuss on the jira the additional caveat of failed metadata fetches.
i.e., since that happens in the network-client the true record error rate would be higher
than what's counted by sendermetrics.

The options that we have are:
* Expose Sender's maybeRegisterTopicMetrics and use that in NetworkClient maybeUpdateMetadata
if there are no known partitions for a topic
* Keep it as you have it for now and just accept the above discrepancy - (or we could address
that in a separate jira as it is orthogonal).
{code}

> KafkaException was not recorded at the per-topic metrics
> --------------------------------------------------------
>
>                 Key: KAFKA-1800
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1800
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Guozhang Wang
>            Assignee: Guozhang Wang
>             Fix For: 0.9.0
>
>         Attachments: KAFKA-1800.patch
>
>
> When KafkaException was thrown from producer.send() call, it is not recorded on the per-topic
record-error-rate, but only the global error-rate.
> Since users are usually monitoring on the per-topic metrics, loosing all dropped message
counts at this level that are caused by kafka producer thrown exceptions such as BufferExhaustedException
could be very dangerous.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message