aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aurora ReviewBot <wfar...@apache.org>
Subject Re: Review Request 47614: Improving scheduling loop performance.
Date Thu, 19 May 2016 21:23:31 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47614/#review134038
-----------------------------------------------------------


Ship it!




Master (9ad153f) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On May 19, 2016, 9:07 p.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47614/
> -----------------------------------------------------------
> 
> (Updated May 19, 2016, 9:07 p.m.)
> 
> 
> Review request for Aurora and Joshua Cohen.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> The recent resource management refactoring resulted in degrading scheduling loop performance
by 4x-6x (depending on cluster state). While it's not noticable in smaller clusters, it turned
out to be a much bigger problem for larger ones (especially for capacity constrained). 
> 
> This patch caches `ResourceBag` calculations for a task (`ResourceRequest`) and offers
(`HostOffer`) participating in a scheduling round. That resulted in improving perf even over
the old "before refactoring" version:
> 
> Before refactoring:
> ```
> Benchmark                                                                      Mode 
Cnt     Score     Error  Units
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt 
 10  2410.352 ± 226.488  ops/s
> ```
> 
> Master:
> ```
> Benchmark                                                                      Mode 
Cnt    Score    Error  Units
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt 
 10  536.645 ± 19.293  ops/s
> ```
> 
> This patch:
> ```
> Benchmark                                                                      Mode 
Cnt     Score    Error  Units
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt 
 10  3051.379 ± 81.867  ops/s
> ```
> 
> This patch has also been validated in prod with matching results: ~25% better perf compared
to "before refactoring" build.
> 
> 
> Diffs
> -----
> 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java f6c8e9f22f374c53ed9ec27818b4bb6df4cd0b42

>   src/main/java/org/apache/aurora/scheduler/HostOffer.java 53fd68c6622d5cb06c0fc3efe002c11fbb9d08c5

>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c324f5957ed3e7b83c082aadd11d7c572d782b77

>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1daf2962bb732d02afa4385106eec22842d8ed60

>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 53b6aee3ba20ad48f51ab783f6f6aeb40e513f45

>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java b0c6070ec0562c951ca7046cb41f93679a5beeb2

>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7c8079fa3f144ee76c6f217fa08fbcca7cf51f2b

>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
29d4d64f05acbbd2c199b094487f3baa748d5f05 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 80126e2ec8b90ffb4c5cd4a8c140da04009f2b6f

>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
88beda79c7112e6ce9e0a7d157f0b819fb7e4b7a 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java 4c447b27052d4c253909fc74269d71efcdf89d29

>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 50d942e48bf2080629b3a8eeb678ee42099ee1b4

> 
> Diff: https://reviews.apache.org/r/47614/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew -Pq build
> e2e
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message