kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jayesh Thakrar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
Date Thu, 05 May 2016 20:26:12 GMT
Jayesh Thakrar created KAFKA-3663:

             Summary: Proposal for a kafka broker command - kafka-brokers.sh
                 Key: KAFKA-3663
                 URL: https://issues.apache.org/jira/browse/KAFKA-3663
             Project: Kafka
          Issue Type: Improvement
          Components: admin
            Reporter: Jayesh Thakrar

This is a proposal for an admin tool - say, kafka-brokers.sh to provide broker related useful
information. Note that I could not see an option to create a child page at https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.

So here are the details for the proposals.

Some of the key succcess factor for Kafka's success are its performant architecture and operational
simplicity. This is further complemented with a set of commandline tools and utilities for
managing topics as well as testing/stress-testing. However currently Kafka lacks commands/tools
to get a cluster and broker overview. Although it should be mentioned that Kafka does expose
cluster information via API and broker metrics via JMX.

*Proposed Change*
This KIP is for a command, say kafka-brokers.sh that provides useful cluster and broker information.

The command will essentially provide the following pieces of information:

* Cluster Overview Information
** Controller Broker Id (and version/epoch information)
** Broker Count
** Total Topic Count
** Total Partition Count

* Broker Information
** Broker Id
** Rack Id
** Hostname
** Endpoints (protocol, port)
** JMX port
** Topic count
** All partition count
** Leader partition count
** Under-replicated partition count
** Topic partitions (Name, *partition-) - 
An asterisk would indicate that the broker is the leader for that partition.
A hyphen/negative sign would indicates that the partition is not in-sync.

As you can see, the above information provides a view of the cluster and brokers that complements

This command can be further evolved to do more things like:
- Drain one or more brokers for decommissioning. This feature would allow distributing off
all partitions of a list of brokers to other brokers in the cluster (need to be cognizant
of rack configuration). 
- Onboard one or more new brokers. This feature would do the reverse of removing brokers and
allow controlled distribution of partition to the new brokers.
- Obtain JMX metrics from a broker (see KIP for its usage).
- See KAFKA-3649 for other features

This message was sent by Atlassian JIRA

View raw message