storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bijoy deb <bijoy.comput...@gmail.com>
Subject Re: Deactivate topology incase of network issues!!!
Date Mon, 10 Feb 2014 08:44:07 GMT
Hi,

You can catch the exception in your bolt (which is connecting to sql
server) and in the catch block you can kill or deactivate the topology
using Nimbus client,as below:

NimbusClient client = .......

client.killTopology("topologyName");//or, client.deactivate("topologyName");



Thanks

Bijoy


On Mon, Feb 10, 2014 at 1:42 PM, Chitra Raveendran <
chitra.raveendran@flutura.com> wrote:

> Hi
>
> I have a Realtime data crunching pipeline which wherein storm consumes
> data from kafka, we do some processing on this data (basically counts and
> aggregations) and store the results into a MySql database.( I use the
> default Kafka spout.)
>
> Over the weekend some network related issues led to MySQL server crash, as
> a result storm kept re-sending the messages, the processing in the storm
> supervisors increased drastically and CPU usage hit 100%. This in turn led
> to some supervisors falling out of the cluster and trying to restart.
>
> How would I be able to *handle* such an unforeseen situation? Is there
> any way through which storm would understand that there is a MySql server
> issue and stop re-sending data?
>
> Is there anyway that this exception can be caught and in *topology be
> deactivated* for a while?
>
> Regards,
> Chitra
>
>

Mime
View raw message