aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Shirchenko <cald...@gmail.com>
Subject Re: Review Request 56690: [Patch 1/2] RFC for first patch implementing scheduling changes for Dynamic Reservations
Date Fri, 24 Feb 2017 00:26:19 GMT


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java, lines
99-100
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634087#file1634087line99>
> >
> >     To short-circuit both loops this should probably be a return.

Done.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/resources/org/apache/aurora/scheduler/tiers.json, lines 29-33
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634095#file1634095line29>
> >
> >     Shoudn't reservations be always backed by quota to prevent abuse?

we don't use Aurora's built-in quota management, instead, relying on our own so all jobs regardless
of tier do not run as production. i can delete this tier as i think these are for reference.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/TierInfo.java, lines 67-69
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634080#file1634080line67>
> >
> >     Copy paste error

Done.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/base/InstanceKeys.java, lines 51-63
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634082#file1634082line51>
> >
> >     `IAssignedTask` has a `task` which in turn has a jobkey. There should therfore
be no need to parse this from the Mesos `TaskInfo`.

Done.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java, lines 213-231
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634094#file1634094line213>
> >
> >     This reads like as if there is a more general concept of "soft-constraints"
waiting to be discovered. In other words: We want constraints that are lifted once we cannot
satisfy them for a certain amount of time. See https://issues.apache.org/jira/browse/AURORA-173
for details.
> >     
> >     I imagine the TaskAssigner could keep its current behavior and only assign a
task if there is no veto for it. It is the responsibility of an out-of-band mechanism to ensure
we either don't generate vetos for expired soft-constraints, or we have a way to check if
a generated veto is already expired or not. 
> >     
> >     (One very rudimentary implementation could probably look similar to `TaskTimeout.java`
but for the `PENDING` state. However there are most likely also other/better ways to do this)

Zameer had thoughts on this.^


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, lines 405-406
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634086#file1634086line405>
> >
> >     Please add a doc string. I may not have completely understood why this is absolutely
needed. 
> >     
> >     We should have a very good reason to break the existing interface.

Done.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java, line 169
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634094#file1634094line169>
> >
> >     Please make sure you always use the built-in formatting of the logger methods.
Otherwise we will assemble strings in memory, only to be discarded by the logger afterwards
:-)

Done.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java, line 211
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634094#file1634094line211>
> >
> >     Same here (and in some other places)

Done. Hopefully I caught all of the places.


> On Feb. 22, 2017, 11:31 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/TierManager.java, lines 100-105
> > <https://reviews.apache.org/r/56690/diff/2/?file=1634081#file1634081line100>
> >
> >     In addition to my concerns about the mutable tier config, the `copyOf` might
come with a performance tax.
> >     
> >     `getTier` is on a hot code path and I therefore recently rewrote the `checkArgument`
here to only construct the String if the argument is invalid. This yielded a significant perf
improvement in our scheduling benchmarks.

Yeah, I could not think of an alternative and am not very happy with this approach myself.
Would love to brainstorm if anyone has better ideas.


- Dmitriy


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56690/#review166430
-----------------------------------------------------------


On Feb. 15, 2017, 1:44 a.m., Dmitriy Shirchenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56690/
> -----------------------------------------------------------
> 
> (Updated Feb. 15, 2017, 1:44 a.m.)
> 
> 
> Review request for Aurora, Mehrdad Nurolahzade, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1819
>     https://issues.apache.org/jira/browse/AURORA-1819
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This is an RFC (without tests) for dynamic reservations proposal. If there is consensus
on the approach, I will add tests. This patch was also tested locally and works as expected.
> 
> 
> Diffs
> -----
> 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java f2296a9d7a88be7e43124370edecfe64415df00f

>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java 6f2ca35c5d83dde29c24865b4826d4932e96da80

>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 676dfd9f9d7ee0633c05424f788fd0ab116976bb

>   src/main/java/org/apache/aurora/scheduler/TierInfo.java c45b949ae7946fc92d7e62f94696ddc4f0790cfa

>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca

>   src/main/java/org/apache/aurora/scheduler/base/InstanceKeys.java b12ac83168401c15fb1d30179ea8e4816f09cd3d

>   src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 0136afb8f6049a6d88cd42b5e3f17d61fcd629d5

>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49

>   src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java f6c759f03c4152ae93317692fc9db202fe251122

>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bb1a960a4c77f48b0ceaa213bd27546551f384f9

>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 60097d91d836e2686d6e90571f13a2fbfd88ae14

>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 0d639f66db456858278b0485c91c40975c3b45ac

>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 8c000cb0626bd34f6f30e23fe2b3a045f2b44e35

>   src/main/java/org/apache/aurora/scheduler/offers/OfferSettings.java e16e36ed360ef9ca371df9084365ea88cfb6e7ce

>   src/main/java/org/apache/aurora/scheduler/offers/OffersModule.java 202cae96ffc5b49e638b973a273f7983137b5baf

>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java 9aa263a9cfae03a9a0c5bc7fe3a1405397d3009c

>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 203f62bacc47470545d095e4d25f7e0f25990ed9

>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java da378e84ee65a658ff2382489d3ab6d5f6451b5f

>   src/main/resources/org/apache/aurora/scheduler/tiers.json 34ddb1dc769a73115c209c9b2ee158cd364392d8

>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840

>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 49d4e82cc03144b80292fe43066a6cc4d7aed88f

>   src/test/java/org/apache/aurora/scheduler/resources/AcceptedOfferTest.java dded9c34749cf599d197ed312ffb6bf63b6033f1

>   src/test/java/org/apache/aurora/scheduler/resources/ResourceManagerTest.java b8b8edb1a21ba89b8b60f8f8451c8c776fc23ae8

>   src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java e04f6113c43eca4555ee0719f8208d7c4ebb8d61

>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java cf2d25ec2e407df7159e0021ddb44adf937e1777

> 
> Diff: https://reviews.apache.org/r/56690/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Dmitriy Shirchenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message