aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Khutornenko <ma...@apache.org>
Subject Review Request 47614: Improving scheduling loop performance.
Date Thu, 19 May 2016 21:07:18 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47614/
-----------------------------------------------------------

Review request for Aurora and Joshua Cohen.


Repository: aurora


Description
-------

The recent resource management refactoring resulted in degrading scheduling loop performance
by 4x-6x (depending on cluster state). While it's not noticable in smaller clusters, it turned
out to be a much bigger problem for larger ones (especially for capacity constrained). 

This patch caches `ResourceBag` calculations for a task (`ResourceRequest`) and offers (`HostOffer`)
participating in a scheduling round. That resulted in improving perf even over the old "before
refactoring" version:

Before refactoring:
```
Benchmark                                                                      Mode  Cnt 
   Score     Error  Units
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10 
2410.352 ± 226.488  ops/s
```

Master:
```
Benchmark                                                                      Mode  Cnt 
  Score    Error  Units
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10 
536.645 ± 19.293  ops/s
```

This patch:
```
Benchmark                                                                      Mode  Cnt 
   Score    Error  Units
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10 
3051.379 ± 81.867  ops/s
```

This patch has also been validated in prod with matching results: ~25% better perf compared
to "before refactoring" build.


Diffs
-----

  src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java f6c8e9f22f374c53ed9ec27818b4bb6df4cd0b42

  src/main/java/org/apache/aurora/scheduler/HostOffer.java 53fd68c6622d5cb06c0fc3efe002c11fbb9d08c5

  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c324f5957ed3e7b83c082aadd11d7c572d782b77

  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1daf2962bb732d02afa4385106eec22842d8ed60

  src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 53b6aee3ba20ad48f51ab783f6f6aeb40e513f45

  src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java b0c6070ec0562c951ca7046cb41f93679a5beeb2

  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7c8079fa3f144ee76c6f217fa08fbcca7cf51f2b

  src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 29d4d64f05acbbd2c199b094487f3baa748d5f05

  src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 80126e2ec8b90ffb4c5cd4a8c140da04009f2b6f

  src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 88beda79c7112e6ce9e0a7d157f0b819fb7e4b7a

  src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java 4c447b27052d4c253909fc74269d71efcdf89d29

  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 50d942e48bf2080629b3a8eeb678ee42099ee1b4


Diff: https://reviews.apache.org/r/47614/diff/


Testing
-------

./gradlew -Pq build
e2e


Thanks,

Maxim Khutornenko


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message