Mailing-List: contact reviews-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: reviews@aurora.apache.org
Content-Type: multipart/alternative;
 boundary="===============6171628346989617860=="
MIME-Version: 1.0
Subject: Re: Review Request 59039: Add the ability to customize scheduling
 logic.
From: David McLaughlin <david@dmclaughlin.com>
To: Santhosh Kumar Shanmugham <santhoshkumar.s@gmail.com>,
 Stephan Erb <serb@apache.org>, Zameer Manji <zmanji@apache.org>
Cc: Joshua Cohen <jcohen@apache.org>, Aurora ReviewBot <wfarner@apache.org>,
 David McLaughlin <david@dmclaughlin.com>, Aurora <reviews@aurora.apache.org>
Date: Tue, 23 May 2017 04:23:46 -0000
Message-ID: <20170523042346.58535.79004@reviews-vm2.apache.org>
Auto-Submitted: auto-generated
Sender: David McLaughlin <noreply@reviews.apache.org>
References: <20170512142002.53986.70182@reviews-vm2.apache.org>
In-Reply-To: <20170512142002.53986.70182@reviews-vm2.apache.org>
Reply-To: David McLaughlin <david@dmclaughlin.com>
archived-at: Tue, 23 May 2017 04:23:52 -0000

--===============6171628346989617860==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> >     Changes like these and others in this patch could potentially break our benchmarking scripts.
> >     
> >     Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
> >     
> >     We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
> 
> Joshua Cohen wrote:
>     Not directly related here, but should we add something that regularly runs the benchmarks and alerts if they stop working (or if there's a major regression in terms of perf)?
> 
> Stephan Erb wrote:
>     Good idea! We might need some tweaking of the benchmark footprint but that should be feasible.
> 
> David McLaughlin wrote:
>     Do the SchedulingBenchmarks really pass for you on master? I spent a while trying to debug the broken scheduling benchmarks.. but going back a month of commits they seem broken then too. This issue is making it really difficult to figure out the missing guice bindings: https://github.com/google/guice/issues/757

AFAICT, they've been broken since this commit: https://github.com/apache/aurora/commit/c32f14c75e7f8620aedb12d38aee8ca1d2427724

I'll try and fix them in a separate patch.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------


On May 11, 2017, 9:46 p.m., David McLaughlin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
> 
> (Updated May 11, 2017, 9:46 p.m.)
> 
> 
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1920
>     https://issues.apache.org/jira/browse/AURORA-1920
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
> 
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07 
>   docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172 
>   src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882 
> 
> 
> Diff: https://reviews.apache.org/r/59039/diff/2/
> 
> 
> Testing
> -------
> 
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
> 
> 
> Thanks,
> 
> David McLaughlin
> 
>


--===============6171628346989617860==--