storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chitra Raveendran <chitra.raveend...@flutura.com>
Subject Re: Deactivate topology incase of network issues!!!
Date Mon, 10 Feb 2014 09:53:45 GMT
Thanks a lot Bijoy

I will test that out and get back to you.

Regards
Chitra


On Mon, Feb 10, 2014 at 2:14 PM, bijoy deb <bijoy.computers@gmail.com>wrote:

> Hi,
>
> You can catch the exception in your bolt (which is connecting to sql
> server) and in the catch block you can kill or deactivate the topology
> using Nimbus client,as below:
>
> NimbusClient client = .......
>
> client.killTopology(
> "topologyName");//or, client.deactivate("topologyName");
>
>
>
> Thanks
>
> Bijoy
>
>
> On Mon, Feb 10, 2014 at 1:42 PM, Chitra Raveendran <
> chitra.raveendran@flutura.com> wrote:
>
>> Hi
>>
>> I have a Realtime data crunching pipeline which wherein storm consumes
>> data from kafka, we do some processing on this data (basically counts and
>> aggregations) and store the results into a MySql database.( I use the
>> default Kafka spout.)
>>
>> Over the weekend some network related issues led to MySQL server crash,
>> as a result storm kept re-sending the messages, the processing in the storm
>> supervisors increased drastically and CPU usage hit 100%. This in turn led
>> to some supervisors falling out of the cluster and trying to restart.
>>
>> How would I be able to *handle* such an unforeseen situation? Is there
>> any way through which storm would understand that there is a MySql server
>> issue and stop re-sending data?
>>
>> Is there anyway that this exception can be caught and in *topology be
>> deactivated* for a while?
>>
>> Regards,
>> Chitra
>>
>>
>

Mime
View raw message