aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Farner" <wfar...@apache.org>
Subject Re: Review Request 27947: Setting the max thread limit on AsyncEventBus.
Date Thu, 13 Nov 2014 03:55:47 GMT


> On Nov. 13, 2014, 2:07 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/events/PubsubEventModule.java, line 93
> > <https://reviews.apache.org/r/27947/diff/1/?file=760831#file760831line93>
> >
> >     Given that we're moving from an effective "same thread executor" to async, i
think we can get by with fewer knobs for now.  How about:
> >     
> >         Executors.newFixedThreadPool(
> >             PUBSUB_WORKER_THREADS.get(),
> >             ...
> 
> Maxim Khutornenko wrote:
>     That was my first choice but ulimately decided in favor of the current approach to
avoid a fixed memory tax of 1Mb X num_threads. With the current setting of 500 that would
result in a constant "waste" of 500MB of JVM heap even when there is no event pressure.

Good point. I'd actually suggest reducing the thread count dramatically, though, maybe 4 by
default.  We've been doing well with 1 for a long time, and throughput isn't driving the need
for the async bus.

I also wonder if we should not use `SynchronousQueue`.  Since we're bounding the number of
threads, busy threads means we'll force the work producer to wait for an available consumer.
 I think in that case we'd rather queue the work item and let the submitter proceed.  What's
your take?


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27947/#review61178
-----------------------------------------------------------


On Nov. 13, 2014, 1:12 a.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27947/
> -----------------------------------------------------------
> 
> (Updated Nov. 13, 2014, 1:12 a.m.)
> 
> 
> Review request for Aurora and Bill Farner.
> 
> 
> Bugs: AURORA-926
>     https://issues.apache.org/jira/browse/AURORA-926
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Setting the max thread limit on AsyncEventBus.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEventModule.java 24cd750c676a280d1cc953c96df126f19fc478f2

> 
> Diff: https://reviews.apache.org/r/27947/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message