storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 唐 思成 <jadetan...@qq.com>
Subject Re: storm performance tunning
Date Sat, 05 Jul 2014 23:26:28 GMT
that really helps,thx a lot
在 2014年7月5日,下午8:35,Eugene <edvorkin@gmail.com> 写道:

> The best tool for topology tuning is Storm UI. Here you can learn your process latency
and capacity of your storm components. 
> Start with some baseline, for example, having all parallelism setting to 1. Then gradually
increase parallelism until you see capacity numbers well below 1.0.
> In general, try to increase number of executors first, before increasing number of workers.
When increasing number of executors (threads) does not give any 
> improvements, then start adding additional workers. Also, if your components more IO
bound then increasing number of task helps as well.
> 
> In addition to Storm UI,and resources Francesco pointed out,  you can use storm metrics
API to send data to graphite. http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/
> 
> Upcoming book, Storm Applied  has a chapter about tuning Storm (chapter 6). 
> 
> Thanks
> Eugene Dvorkin
> 
> 
> 
> 
> 
> 
> 
> On Sat, Jul 5, 2014 at 7:19 AM, <francesco.masucci@pronetics.it> wrote:
> i ' ve found some good advices reading these articles
> 
> http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/
> 
> http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/

> 
>  
>  
>  
> Il 2014-07-05 12:12 唐思成 ha scritto:
> 
>> everyone, do you have some best practice or pattern for tunning the storm cluster,
like how to set the number of workers, the numbers of spout and bolts, etc..
>> Any advice is welcome.
>>  
>> 2014-07-05
>> 唐思成
>  
>  
> 
> 
> 
> -- 
> Eugene Dvorkin
> Software Engineer
> New York City Storm User Group - organizer
> WebMD
> email: edvorkin@gmail.com
> phone: 646-250-9649
> eugenedvorkin.com
> Connect with me on: 
> LinkedIn Twitter


Mime
View raw message