kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
Date Tue, 04 Nov 2014 16:41:35 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196328#comment-14196328
] 

Pradeep commented on KAFKA-1738:
--------------------------------

yes.. This is happening in 8.2.0-beta version which we are currently on. I had two topic in
kakfa before killing the service and i added three more topics (ATopic, BTopic and CTopic)
and all of them got created successfully as you can see below. When i list it i can see the
topics in kakfa. But if i see the data folder in kafka, i dont see the partition folders/files
for them. Let me know if you need more details.

[root@dmipvm temp]# service kafka status
Kafka is running as 19396.
    LISTEN on tcp port=9999
    LISTEN on tcp port=53536
    LISTEN on tcp port=9092
    LISTEN on tcp port=48330
[root@dmipvm temp]# kill -9 19396
[root@dmipvm temp]# service kafka start
Starting kafka ... STARTED.
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic ATopic --partitions 1
--replication-factor 1 --zookeeper localhost:2181
Created topic "ATopic".
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic BTopic --partitions 1
--replication-factor 1 --zookeeper localhost:2181
Created topic "BTopic".
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic CTopic --partitions 1
--replication-factor 1 --zookeeper localhost:2181
Created topic "CTopic".
[root@dmipvm temp]#


[root@DMIPVM kafka]# ls -lrt
total 16
drwxr-xr-x 2 root root 4096 Nov  4 11:32 topic_1-0
drwxr-xr-x 2 root root 4096 Nov  4 11:34 topic_2-0
-rw-r--r-- 1 root root   28 Nov  4 11:35 replication-offset-checkpoint
-rw-r--r-- 1 root root   28 Nov  4 11:36 recovery-point-offset-checkpoint


[root@DMIPVM kafka]# /apps/kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181
ATopic
BTopic
CTopic
topic_1
topic_2


> Partitions for topic not created after restart from forced shutdown
> -------------------------------------------------------------------
>
>                 Key: KAFKA-1738
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1738
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8.1.1, 0.8.2
>         Environment: Linux, 2GB RAM, 2 Core CPU
>            Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the topic gets
created but we don't see the partition specific files and when producer/consumer tries to
get the topic metadata and it fails with exception. Same happens if one tries to create using
the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for topic tloader1
-> No partition metadata for topic tloader1 due to kafka.common.UnknownTopicOrPartitionException}]
for topic [tloader1]: class kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce - 
> 1.      Stop kafka using kill  -9 <PID of Kafka>
> 2.      Start Kafka
> 3.      Create Topic with partition and replication factor of 1.
> 4.      Check the response “Created topic <topic_name>”
> 5.      Run the list command to verify if its created.
> 6.      Now check the data directory of kakfa. There would not be any for the newly created
topic.
> We see issues when we are creating new topics. This happens randomly and we dont know
the exact reasons. We see the below logs in controller during the time of creation of topics
which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for [JobJTopic,0]
(kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation callback for [JobJTopic,0]
(kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]: Invoking state
change to NewPartition for partitions [JobJTopic,0] (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]: Invoking state
change to NewReplica for replicas [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]: Invoking state
change to OnlinePartition for partitions [JobJTopic,0] (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]: Live assigned
replicas for partition [JobJTopic,0] are: [List(0)] (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]: Initializing
leader and isr for partition [JobJTopic,0] to (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
(kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]: Invoking state
change to OnlineReplica for replicas [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread], Controller 0 fails
to send a request to broker id:0,host:DMIPVM,port:9092 (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
> 	at kafka.utils.Utils$.read(Utils.scala:381)
> 	at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> 	at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> 	at kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> 	at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> 	at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> 	at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread], Controller 0
epoch 2 failed to send request Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
-> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker. (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
> 	at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
> 	at kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
> 	at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
> 	at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message