storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <roshan_n...@yahoo.com>
Subject Re: Storm 2.0 and memory for worker
Date Sat, 10 Aug 2019 05:45:41 GMT
  
I think the change in behavior has to do with the resource aware scheduler.  I have noticed
this setting impact the heap size. 
- topology.component.resources.onheap.memory.mb 

It (i think) allocates that much mem to each component instance running in any given worker.
So each worker's heap can be diff. 
If RAS is disabled, I suspect the behavior *might* revert to what you are used to 1.x. Perhaps
someone better familiar with RAS can comment.

-roshan



On Thursday, August 1, 2019, 01:23:53 AM PDT, Stig Rohde Døssing <stigdoessing@gmail.com>
wrote: 





Here are a couple of links to the source that might help you.

The system property is coming from https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L574

It is being calculated by https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L805

See also https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L579

Den ons. 31. jul. 2019 kl. 18.41 skrev Michal Koziorowski <Michal.Koziorowski@tomtom.com>:
>  
>  
>  Hi,
> 
>  
> 
> 
>  In storm 1.1 I could set Xmx for a worker by setting worker.heap.memory.mb in storm.yaml.
> 
>  It seems ignored in my upgraded Storm 2.0 configuration and Xmx value was taken from -Dworker.memory_limit_mb=5888
which I didn't set.
> 
>  How to make setting from storm 1.1 working in 2.0?
> 
>  
> 
> 
>  I have following properties in storm.yaml:
> 
>  worker.heap.memory.mb: 16000
> 
> 
>  worker.childopts: "-Xmx%HEAP-MEM%m -Xlog:gc*,gc+age=trace,safepoint:file=artifacts/gc.log:utctime,pid,tags:filecount=32,filesize=10m
-Djava.net.preferIPv4Stack=true"
> 
> 
>  
> 
> 
>  Previously 16000 was used as %HEAP-MEM%
> 
>  
> 
> 
> 

Mime
View raw message