kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Humbarger (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-8103) Kafka SIGSEGV on kafka-network-thread
Date Wed, 13 Mar 2019 16:28:00 GMT
Sean Humbarger created KAFKA-8103:
-------------------------------------

             Summary: Kafka SIGSEGV on kafka-network-thread
                 Key: KAFKA-8103
                 URL: https://issues.apache.org/jira/browse/KAFKA-8103
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 1.1.1
         Environment: OS 
{code}
Amazon Linux
{code}

Kernel 
{code}
4.14.97-74.72.amzn1.x86_64 #1 SMP Tue Feb 5 20:59:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
{code}

Java
{code}
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
{code}

AWS Instance Type
{code}
c5.4xlarge
{code}
            Reporter: Sean Humbarger
         Attachments: hs_err_pid4345.log

We have a 4 node cluster (6 topics, 6 consumer groups) that is processing 65,000 messages
per second and are seeing SIGSEGV crashes at least once a day (see attachment).  Each broker
has six disks attached to it to support the kafka logs.  When the crash occurs, we simply
restart kafka and everything seems fine.  We don't see any out of the ordinary in /var/log/messages
or dmesg when the crashes occur.  Thus far, we are unable to predict during the day when
the crash will occur or which node it will occur on. 

 

The problematic frame is as follows:
{code}

# Problematic frame:
# J 8628 C2 org.apache.kafka.common.metrics.stats.Max.update(Lorg/apache/kafka/common/metrics/stats/SampledStat$Sample;Lorg/apache/kafka/common/metrics/MetricConfig;DJ)V
(13 bytes) @ 0x00007ff779f9fca0 [0x00007ff779f9fc80+0x20]
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message