storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chitra Raveendran <chitra.raveend...@flutura.com>
Subject Deactivate topology incase of network issues!!!
Date Mon, 10 Feb 2014 08:12:28 GMT
Hi

I have a Realtime data crunching pipeline which wherein storm consumes data
from kafka, we do some processing on this data (basically counts and
aggregations) and store the results into a MySql database.( I use the
default Kafka spout.)

Over the weekend some network related issues led to MySQL server crash, as
a result storm kept re-sending the messages, the processing in the storm
supervisors increased drastically and CPU usage hit 100%. This in turn led
to some supervisors falling out of the cluster and trying to restart.

How would I be able to *handle* such an unforeseen situation? Is there any
way through which storm would understand that there is a MySql server issue
and stop re-sending data?

Is there anyway that this exception can be caught and in *topology be
deactivated* for a while?

Regards,
Chitra

Mime
View raw message