aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McLaughlin <da...@dmclaughlin.com>
Subject Re: Review Request 59039: Add the ability to customize scheduling logic.
Date Tue, 23 May 2017 04:23:46 GMT


> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> >     Changes like these and others in this patch could potentially break our benchmarking
scripts.
> >     
> >     Could you please make sure that at least the `SchedulingBenchmarks` continue
to run?
> >     
> >     We already have several broken the benchmarks suites `ThriftApiBenchmarks`,
`SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if
we break even more.
> 
> Joshua Cohen wrote:
>     Not directly related here, but should we add something that regularly runs the benchmarks
and alerts if they stop working (or if there's a major regression in terms of perf)?
> 
> Stephan Erb wrote:
>     Good idea! We might need some tweaking of the benchmark footprint but that should
be feasible.
> 
> David McLaughlin wrote:
>     Do the SchedulingBenchmarks really pass for you on master? I spent a while trying
to debug the broken scheduling benchmarks.. but going back a month of commits they seem broken
then too. This issue is making it really difficult to figure out the missing guice bindings:
https://github.com/google/guice/issues/757

AFAICT, they've been broken since this commit: https://github.com/apache/aurora/commit/c32f14c75e7f8620aedb12d38aee8ca1d2427724

I'll try and fix them in a separate patch.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------


On May 11, 2017, 9:46 p.m., David McLaughlin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
> 
> (Updated May 11, 2017, 9:46 p.m.)
> 
> 
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1920
>     https://issues.apache.org/jira/browse/AURORA-1920
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit
preemption logic.
> 
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07 
>   docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java
PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java
PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172

>   src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION

>   src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57

>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882

> 
> 
> Diff: https://reviews.apache.org/r/59039/diff/2/
> 
> 
> Testing
> -------
> 
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> I've used this in one of our large-scale test clusters to plug in a custom task assigner.
We were able to remove our modifications to the OSS code-base and use this mechanism to do
what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
> 
> 
> Thanks,
> 
> David McLaughlin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message