storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Landman <adrian.land...@gmail.com>
Subject Bolt vs Spout
Date Mon, 28 Jul 2014 16:54:37 GMT
I am writing a topology that pulls messages from a topic, does some work,
and then writes them back on a different topic.  I have been having some
issues so I created my own small topology that just pulls a message, prints
the contents, and then stores them back on a new topic.  I finally got this
to work, but it raised a question.

To create the spout I need to either pass in the kafka location sans port
(e.g. localhost) or use 2181 as the port.

To create the producer bolt I need to pass in the broker port (e.g. 9092)
or I get an array out of bounds exception when creating the Producer.

When we were using kafka7 and
https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka/src/jvm/storm/kafka
for our storm/kafka integration I believe that we used the same broker list
for both our spout and our producer.  Is there anyway to do the same with
kafka8 and the new storm-kafka project?  Also, if w want to pass in a list,
I know that in the KafkaBolt we can set metadata.broker.list to a comma
separated list of brokers (1.1.1.1:9092, 1.1.1.2:9092) but can we do the
same for the spout?  Or is there any reason to?  ZkHost takes in a String,
but I didn't see anything that specified the format.

Mime
View raw message