kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-2134) Producer blocked on metric publish
Date Mon, 03 Aug 2015 02:27:04 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651343#comment-14651343

ASF GitHub Bot commented on KAFKA-2134:

GitHub user becketqin opened a pull request:


    Fix for KAFKA-2134, fix replica offset truncate to beginning during leader migration.

    Fix replica truncate log to beginning during leader migration.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/becketqin/kafka KAFKA-2134

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #104
commit 71f8a4716e1f0b4fc2bd88aa30fe38aef8a9f92e
Author: Jiangjie Qin <becket.qin@gmail.com>
Date:   2015-08-03T02:22:02Z

    Fix for KAFKA-2134, fix replica offset truncate to beginning during leader migration.


> Producer blocked on metric publish
> ----------------------------------
>                 Key: KAFKA-2134
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2134
>             Project: Kafka
>          Issue Type: Bug
>          Components: producer 
>    Affects Versions:
>         Environment: debian7, java8
>            Reporter: Vamsi Subhash Achanta
>            Assignee: Jun Rao
>            Priority: Blocker
> Hi,
> We have a REST api to publish to a topic. Yesterday, we started noticing that the producer
is not able to produce messages at a good rate and the CLOSE_WAITs of our producer REST app
are very high. All the producer REST requests are hence timing out.
> When we took the thread dump and analysed it, we noticed that the threads are getting
blocked on JmxReporter metricChange. Here is the attached stack trace.
> "dw-70 - POST /queues/queue_1/messages" #70 prio=5 os_prio=0 tid=0x00007f043c8bd000 nid=0x54cf
waiting for monitor entry [0x00007f04363c7000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
>         at org.apache.kafka.common.metrics.JmxReporter.metricChange(JmxReporter.java:76)
>         - waiting to lock <0x00000005c1823860> (a java.lang.Object)
>         at org.apache.kafka.common.metrics.Metrics.registerMetric(Metrics.java:182)
>         - locked <0x00000007a5e526c8> (a org.apache.kafka.common.metrics.Metrics)
>         at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:165)
>         - locked <0x00000007a5e526e8> (a org.apache.kafka.common.metrics.Sensor)
> When I looked at the code of metricChange method, it uses a synchronised block on an
object resource and it seems that it is held by another.

This message was sent by Atlassian JIRA

View raw message