aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McLaughlin <>
Subject Re: Review Request 58259: Add update affinity to Scheduler
Date Tue, 02 May 2017 01:32:25 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated May 2, 2017, 1:32 a.m.)

Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Attached screenshot of the cache hit rate difference before and after tuning the max tasks
per job and max scheduling penalty (for sustained 1k instances updated per minute across 10

Repository: aurora


In the Dynamic Reservations review (and on the mailing list), I mentioned that we could implement
update affinity with less complexity using the same technique as preemption. Here is how that
would work. 

This just adds a simple wrapper around the preemptor's BiCache structure and then optimistically
tries to keep an agent free for a task during the update process. 

Note: I don't bother even checking the resources before reserving the agent. I figure there
is a chance the agent has enough room, and if not we'll catch it when we attempt to veto the
offer. We need to always check the offer like this anyway in case constraints change. In the
worst case it adds some delay in the rare cases you increase resources. 

We also don't persist the reservations, so if the Scheduler fails over during an update, the
worst case is that any instances between the KILLED and ASSIGNED in-flight batch need to fall
back to the current first-fit scheduling algorithm.


  src/main/java/org/apache/aurora/scheduler/base/ f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49

  src/main/java/org/apache/aurora/scheduler/scheduling/ 203f62bacc47470545d095e4d25f7e0f25990ed9

  src/main/java/org/apache/aurora/scheduler/state/ a177b301203143539b052524d14043ec8a85a46d

  src/main/java/org/apache/aurora/scheduler/updater/ b4cd01b3e03029157d5ca5d1d8e79f01296b57c2

  src/main/java/org/apache/aurora/scheduler/updater/ f25dc0c6d9c05833b9938b023669c9c36a489f68

  src/main/java/org/apache/aurora/scheduler/updater/ c129896d8cd54abd2634e2a339c27921042b0162

  src/main/java/org/apache/aurora/scheduler/updater/ e14112479807b4477b82554caf84fe733f62cf58

  src/main/java/org/apache/aurora/scheduler/updater/ c95943d242dc2f539778bdc9e071f342005e8de3

  src/main/java/org/apache/aurora/scheduler/updater/ PRE-CREATION

  src/main/java/org/apache/aurora/scheduler/updater/ 13cbdadad606d9acaadc541320b22b0ae538cc5e

  src/test/java/org/apache/aurora/scheduler/scheduling/ fa1a81785802b82542030e1aae786fe9570d9827

  src/test/java/org/apache/aurora/scheduler/state/ cf2d25ec2e407df7159e0021ddb44adf937e1777

  src/test/java/org/apache/aurora/scheduler/updater/ b2c4c66850dd8f35e06a631809530faa3b776252

  src/test/java/org/apache/aurora/scheduler/updater/ c78c7fbd7d600586136863c99ce3d7387895efee

  src/test/java/org/apache/aurora/scheduler/updater/ 30b44f88a5b8477e917da21d92361aea1a39ceeb

  src/test/java/org/apache/aurora/scheduler/updater/ 833fd62c870f96b96343ee5e0eed0d439536381f

  src/test/java/org/apache/aurora/scheduler/updater/ PRE-CREATION

  src/test/java/org/apache/aurora/scheduler/updater/ PRE-CREATION



./gradlew build

File Attachments (updated)

Cache utilization over time
Offer rate from Mesos over time
Async task workload (scaled) correlation with degraded cache utilization
cache hit rate before and after scheduler tuning


David McLaughlin

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message