aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxim Khutornenko" <ma...@apache.org>
Subject Re: Review Request 25812: Implementing quota checking for async job updates.
Date Fri, 19 Sep 2014 19:53:02 GMT


> On Sept. 19, 2014, 7:38 p.m., Bill Farner wrote:
> > Do you think it makes sense to wait for AURORA-718 before we land this?  I think
that will help simplify the algorithm:
> > 
> > - convert each job to a resource aggregate
> > - convert each job update to a possibly-negative resource aggregate delta
> > - walk each job, add delta if positive

This diff is built assuming the AURORA-718. As for the algorithm suggestion, I don't see how
it would work for in-flight updates. Once the update started, the baseline for the delta is
gone and we may be double counting (or missing) instances that have already been worked on.


The current algorithm does not suffer from that flaw as it treats affected instances separately:
- convert all instance tasks unaffected by update into resource aggregate
- convert all instance tasks affected by update into resource aggregate
- add the two above to yield overall consumption


- Maxim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25812/#review54000
-----------------------------------------------------------


On Sept. 19, 2014, 6:03 p.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25812/
> -----------------------------------------------------------
> 
> (Updated Sept. 19, 2014, 6:03 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Bill Farner.
> 
> 
> Bugs: AURORA-686
>     https://issues.apache.org/jira/browse/AURORA-686
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Getting it right required a deep refactoring of the QuotaManager. The prod consumption
is now calculated from:
> - production tasks not participating in job updates;
> - unaffected production tasks from a job being updated (i.e. those that are not covered
by the update working set);
> - production tasks directly affected by the job update (max of old and new resources
used).
> 
> Also, moved all logic back to SchedulerThriftInterface as quota checks are done only
there now.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e

>   src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d

>   src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4

>   src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 779e925e4d9e7889e8cfd369cea9a8e5da3554d2

>   src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 83ac034cff009530e5e16c0613b9d085f3b908d8

>   src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518

>   src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 49770e5f87f047502e4f5653b908657a40d8683f

>   src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276

>   src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090

>   src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
336cada625b85618486660fc24f3e83a312609f8 
>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9

> 
> Diff: https://reviews.apache.org/r/25812/diff/
> 
> 
> Testing
> -------
> 
> gradle -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message