storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Webb" <>
Subject Parallelism of IMetricsConsumer
Date Thu, 01 Sep 2016 13:16:33 GMT

I want to gather some Metrics of all Topologies running in Storm and came across the IMetricsConsumer.
In order to add gathering of metrics to all topologies, I added my implementation in the "storm.yaml"

Although it works great so far, I have some conceptual questions.
As far as I understand it, the MetricsConsumer is internally implemented as a Bolt which is
connected to all
other Bolts. As with Bolts, I can adjust the degree of parallelism by specifying a "parallelism.hint"
basically defines the amount of Executors (i.e. Threads) that execute this Bolt.

Question 1:
With which kind of Grouping is the MetricsConsumerBolt connected to the other Bolts? I would
have guessed
"shuffle grouping" or "local or shuffle grouping", is this true?

Question 2:
Despite being able to define the amount of Executors, can I specify in which way I'd like
to "distribute"
the MetricsConsumerBolt "instances".
Imagine that my topology is split across 2 workers which are running on separate machines
and I've set
"parallelism.hint: 2". Where will the two MetricsConsumerBolt Tasks run? One Task on each
Worker? Both Tasks
on one Worker?

Thank you!
Dave Webb

View raw message