kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ismael Juma (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-7625) Kafka Broker node JVM crash - kafka.coordinator.transaction.TransactionCoordinator.$anonfun$handleEndTransaction
Date Thu, 27 Dec 2018 16:00:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-7625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ismael Juma resolved KAFKA-7625.
--------------------------------
    Resolution: Not A Bug

Closing since it's not a Kafka issue. Someone else reported that changing the GC from G1 also
made the issue go away. In any case, upgrading to the latest version of JDK 8 is the recommended
path.

> Kafka Broker node JVM crash - kafka.coordinator.transaction.TransactionCoordinator.$anonfun$handleEndTransaction
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-7625
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7625
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 2.0.0
>         Environment:  environment:os.version=2.6.32-754.2.1.el6.x86_64 
> java.version=1.8.0_92 
> environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built
on 06/29/2018 00:39 GMT (org.apache.zookeeper.ZooKeeper)
> Kafka commitId : 3402a8361b734732 
>            Reporter: Sebastian Puzoń
>            Priority: Critical
>         Attachments: hs_err_pid10238.log, hs_err_pid15119.log, hs_err_pid19131.log, hs_err_pid19405.log,
hs_err_pid20124.log, hs_err_pid22373.log, hs_err_pid22386.log, hs_err_pid22633.log, hs_err_pid24681.log,
hs_err_pid25513.log, hs_err_pid25701.log, hs_err_pid26844.log, hs_err_pid27156.log, hs_err_pid27290.log,
hs_err_pid4194.log, hs_err_pid4299.log
>
>
> I observe broker node JVM crashes with same problematic frame:
> {code:java}
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007ff4a2588261, pid=24681, tid=0x00007ff3b9bb1700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_92-b14) (build 1.8.0_92-b14)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.92-b14 mixed mode linux-amd64 compressed
oops)
> # Problematic frame:
> # J 9736 C1 kafka.coordinator.transaction.TransactionCoordinator.$anonfun$handleEndTransaction$7(Lkafka/coordinator/transaction/TransactionCoordinator;Ljava/lang/String;JSLorg/apache/kafka/common/requests/TransactionResult;Lkafka/coordinator/transaction/TransactionMetadata;)Lscala/util/Either;
(518 bytes) @ 0x00007ff4a2588261 [0x00007ff4a25871a0+0x10c1]
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try
"ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> ---------------  T H R E A D  ---------------
> Current thread (0x00007ff4b356f800):  JavaThread "kafka-request-handler-3" daemon [_thread_in_Java,
id=24781, stack(0x00007ff3b9ab1000,0x00007ff3b9bb2000)]
> {code}
> {code:java}
> Stack: [0x00007ff3b9ab1000,0x00007ff3b9bb2000],  sp=0x00007ff3b9bafca0,  free space=1019k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> J 9736 C1 kafka.coordinator.transaction.TransactionCoordinator.$anonfun$handleEndTransaction$7(Lkafka/coordinator/transaction/TransactionCoordinator;Ljava/lang/String;JSLorg/apache/kafka/common/requests/TransactionResult;Lkafka/coordinator/transaction/TransactionMetadata;)Lscala/util/Either;
(518 bytes) @ 0x00007ff4a2588261 [0x00007ff4a25871a0+0x10c1]
> J 10456 C2 kafka.coordinator.transaction.TransactionCoordinator.$anonfun$handleEndTransaction$5(Lkafka/coordinator/transaction/TransactionCoordinator;Ljava/lang/String;JSLorg/apache/kafka/common/requests/TransactionResult;ILscala/Option;)Lscala/util/Either;
(192 bytes) @ 0x00007ff4a1d413f0 [0x00007ff4a1d41240+0x1b0]
> J 9303 C1 kafka.coordinator.transaction.TransactionCoordinator$$Lambda$1107.apply(Ljava/lang/Object;)Ljava/lang/Object;
(32 bytes) @ 0x00007ff4a245f55c [0x00007ff4a245f3c0+0x19c]
> J 10018 C2 scala.util.Either$RightProjection.flatMap(Lscala/Function1;)Lscala/util/Either;
(43 bytes) @ 0x00007ff4a1f242c4 [0x00007ff4a1f24260+0x64]
> J 9644 C1 kafka.coordinator.transaction.TransactionCoordinator.sendTxnMarkersCallback$1(Lorg/apache/kafka/common/protocol/Errors;Ljava/lang/String;JSLorg/apache/kafka/common/requests/TransactionResult;Lscala/Function1;ILkafka/coordinator/transaction/TxnTransitMetadata;)V
(251 bytes) @ 0x00007ff4a1ef6254 [0x00007ff4a1ef5120+0x1134]
> J 9302 C1 kafka.coordinator.transaction.TransactionCoordinator$$Lambda$1106.apply(Ljava/lang/Object;)Ljava/lang/Object;
(40 bytes) @ 0x00007ff4a24747ec [0x00007ff4a24745a0+0x24c]
> J 10125 C2 kafka.coordinator.transaction.TransactionStateManager.updateCacheCallback$1(Lscala/collection/Map;Ljava/lang/String;ILkafka/coordinator/transaction/TxnTransitMetadata;Lscala/Function1;Lscala/Function1;Lorg/apache/kafka/common/TopicPartition;)V
(892 bytes) @ 0x00007ff4a27045ec [0x00007ff4a2703c60+0x98c]
> J 10051 C2 kafka.coordinator.transaction.TransactionStateManager$$Lambda$814.apply(Ljava/lang/Object;)Ljava/lang/Object;
(36 bytes) @ 0x00007ff4a1a9cd08 [0x00007ff4a1a9cc80+0x88]
> J 9349 C2 kafka.server.DelayedProduce.tryComplete()Z (52 bytes) @ 0x00007ff4a1e46e5c
[0x00007ff4a1e46980+0x4dc]
> J 10111 C2 kafka.server.DelayedOperationPurgatory.tryCompleteElseWatch(Lkafka/server/DelayedOperation;Lscala/collection/Seq;)Z
(147 bytes) @ 0x00007ff4a1c6e000 [0x00007ff4a1c6df20+0xe0]
> J 10448 C2 kafka.server.ReplicaManager.appendRecords(JSZZLscala/collection/Map;Lscala/Function1;Lscala/Option;Lscala/Function1;)V
(237 bytes) @ 0x00007ff4a2340b6c [0x00007ff4a233f3e0+0x178c]
> J 10050 C2 kafka.coordinator.transaction.TransactionStateManager.$anonfun$appendTransactionToLog$13(Lkafka/coordinator/transaction/TransactionStateManager;Ljava/lang/String;ILkafka/coordinator/transaction/TxnTransitMetadata;Lscala/Function1;Lscala/Function1;Lorg/apache/kafka/common/TopicPartition;Lscala/collection/immutable/Map;)V
(294 bytes) @ 0x00007ff4a25a2384 [0x00007ff4a25a1800+0xb84]
> J 10112 C2 kafka.coordinator.transaction.TransactionStateManager$$Lambda$812.apply$mcV$sp()V
(36 bytes) @ 0x00007ff4a1b24e64 [0x00007ff4a1b24de0+0x84]
> J 4800 C2 scala.runtime.java8.JFunction0$mcV$sp.apply()Ljava/lang/Object; (10 bytes)
@ 0x00007ff4a1ad73dc [0x00007ff4a1ad73a0+0x3c]
> J 6365 C2 kafka.utils.CoreUtils$.inReadLock(Ljava/util/concurrent/locks/ReadWriteLock;Lscala/Function0;)Ljava/lang/Object;
(12 bytes) @ 0x00007ff4a13c7794 [0x00007ff4a13c7720+0x74]
> J 10037 C2 kafka.coordinator.transaction.TransactionStateManager.appendTransactionToLog(Ljava/lang/String;ILkafka/coordinator/transaction/TxnTransitMetadata;Lscala/Function1;Lscala/Function1;)V
(152 bytes) @ 0x00007ff4a26cd90c [0x00007ff4a26c9f40+0x39cc]
> J 10447 C2 kafka.server.KafkaApis.handleEndTxnRequest(Lkafka/network/RequestChannel$Request;)V
(118 bytes) @ 0x00007ff4a1fbb154 [0x00007ff4a1fb9fa0+0x11b4]
> J 6316 C2 kafka.server.KafkaApis.handle(Lkafka/network/RequestChannel$Request;)V (1154
bytes) @ 0x00007ff4a19538c0 [0x00007ff4a1953120+0x7a0]
> J 8822% C2 kafka.server.KafkaRequestHandler.run()V (258 bytes) @ 0x00007ff4a1d43bc4 [0x00007ff4a1d43a60+0x164]
> j  java.lang.Thread.run()V+11
> v  ~StubRoutines::call_stub
> {code}
>  
>  Attached crash reports:
>  
> /apps/solr/kafka/kafka_2.12-2.0.0 $ ls -lth hs_err_pid*
> -rwxrwxrwx 1 res2 res 424K Nov 14 14:02 hs_err_pid24681.log
> -rwxrwxrwx 1 res2 res 427K Nov 13 05:18 hs_err_pid4194.log
> -rwxrwxrwx 1 res2 res 423K Nov 12 17:06 hs_err_pid15119.log
> -rwxrwxrwx 1 res2 res 425K Nov 12 04:56 hs_err_pid4299.log
> -rwxrwxrwx 1 res2 res 423K Nov 11 16:32 hs_err_pid25701.log
> -rwxrwxrwx 1 res2 res 424K Nov 11 03:44 hs_err_pid22373.log
> -rwxrwxrwx 1 res2 res 426K Nov 10 15:24 hs_err_pid25513.log
> -rwxrwxrwx 1 res2 res 421K Nov 10 02:16 hs_err_pid10238.log
> -rwxrwxrwx 1 res2 res 435K Nov  8 11:45 hs_err_pid22386.log
> -rwxrwxrwx 1 res2 res 417K Nov  7 20:38 hs_err_pid26844.log
> -rwxrwxrwx 1 res2 res 419K Nov  7 08:04 hs_err_pid27290.log
> -rwxrwxrwx 1 res2 res 419K Nov  6 17:37 hs_err_pid19131.log
> -rwxrwxrwx 1 res2 res 418K Nov  6 03:50 hs_err_pid22633.log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message