ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carita Ou (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-12678) Installing additional Kafka broker on node with hostname that comes alphabetically before existing broker fails to start
Date Fri, 07 Aug 2015 17:41:47 GMT
Carita Ou created AMBARI-12678:
----------------------------------

             Summary: Installing additional Kafka broker on node with hostname that comes
alphabetically before existing broker fails to start
                 Key: AMBARI-12678
                 URL: https://issues.apache.org/jira/browse/AMBARI-12678
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.0.0, 2.1.0
            Reporter: Carita Ou
            Priority: Minor


1. Install Ambari with 3 nodes, example hostname01, hostname02, hostname03
2. Install Kafka service on hostname03. Kafka starts successfully
3. Go to hostname02 and add Kafka Broker. Start kafka broker and it will fail with below error:
{code}
[2015-08-05 17:13:51,306] FATAL Fatal error during KafkaServerStartable startup. Prepare to
shutdown (kafka.server.KafkaServerStartable)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/0. This
probably indicates that you either have configured a brokerid that is already in use, or else
you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears
to be re-registering.
		at kafka.utils.ZkUtils$.registerBrokerInZk(ZkUtils.scala:174)
		at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:58)
		at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:45)
		at kafka.server.KafkaServer.startup(KafkaServer.scala:124)
		at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
		at kafka.Kafka$.main(Kafka.scala:46)
		at kafka.Kafka.main(Kafka.scala)
{code}
The 2nd broker added on hostname02 failed to start because it was assigned the same brokerid
as the broker installed on hostname03. The brokerid is assigned to a broker in kafka.py:
{code}
brokerid = str(sorted(params.kafka_hosts).index(params.hostname))
kafka_server_config['broker.id'] = brokerid
{code}
params.py gets the kafka_hosts from clusterHostInfo:
{code}
kafka_hosts = config['clusterHostInfo']['kafka_broker_hosts']
{code}
clusterHostInfo maintains the list of kafka brokers in order:
{code}
"kafka_broker_hosts": [
	"hostname02",
	"hostname03"
],
{code}
Since the brokerid is the index of the hostname, and the broker on hostname03 was added first,
it was at index 0. After adding a new broker on hostname02, it was inserted to the list at
index 0 and hostname03 was moved to index 1. Because the broker on hostname03 was not restarted,
it was still running with brokerid 0, so the broker on hostname02 with brokerid 0 was unable
to start.

Workaround: Restart all brokers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message