zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhimanyu Nagrath <abhimanyunagr...@gmail.com>
Subject kafka.admin.TopicCommand Failing
Date Thu, 16 Nov 2017 11:38:00 GMT
Hi, I am using a single node Kafka V 0.10.2 (16 GB RAM, 8 cores) and a
single node zookeeper V  3.4.9 (4 GB RAM, 1 core ). I am having 64 consumer
groups and 500 topics each having 250 partitions. I am able to execute the
commands which require only Kafka broker and its running fine
ex.

> ./kafka-consumer-groups.sh --bootstrap-server localhost:9092
> --describe --group <topic>

But when I execute the admin command like create topic, alter topic For
example

> ./kafka-topics.sh --create --zookeeper <zookeeper>:2181
> --replication-factor 1 --partitions 1 --topic <topic>

Following exception is being displayed:



> Error while executing topic command : replication factor: 1 larger
> than available brokers: 0 [2017-11-16 11:22:13,592] ERROR
> org.apache.kafka.common.errors.InvalidReplicationFactorException:
> replication factor: 1 larger than available brokers: 0
> (kafka.admin.TopicCommand$)

I checked my broker is up. In server.log following warnings are there

    [2017-11-16 11:14:26,959] WARN Client session timed out, have not heard
from server in 15843ms for sessionid 0x15aa7f586e1c061
(org.apache.zookeeper.ClientCnxn)
    [2017-11-16 11:14:28,795] WARN Unable to reconnect to ZooKeeper
service, session 0x15aa7f586e1c061 has expired
(org.apache.zookeeper.ClientCnxn)
    [2017-11-16 11:21:46,055] WARN Unable to reconnect to ZooKeeper
service, session 0x15aa7f586e1c067 has expired
(org.apache.zookeeper.ClientCnxn)

Below mentioned is my Kafka server configuration :

    broker.id=1
    delete.topic.enable=true
    num.network.threads=3
    num.io.threads=8
    socket.send.buffer.bytes=102400
    socket.receive.buffer.bytes=102400
    socket.request.max.bytes=104857600
    log.dirs=/kafka/data/logs
    num.partitions=1
    log.segment.bytes=1073741824
    log.retention.check.interval.ms=300000
    zookeeper.connect=<zookeeperIP>:2181
    zookeeper.connection.timeout.ms=6000

Zookeeper Configuration is :

    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    # do not use /tmp for storage, /tmp here is just
    # example sakes.
    dataDir=/zookeeper/data
    # the port at which the clients will connect
    clientPort=2181
    # the maximum number of client connections.
    # increase this if you need to handle more clients
    #maxClientCnxns=60
    autopurge.snapRetainCount=20
    # Purge task interval in hours
    # Set to "0" to disable auto purge feature
    autopurge.purgeInterval=48

I am not able to figure out which configuration to tune. What I am missing
.Any help will be appreciated.




 Regards,
Abhimanyu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message