Mailing-List: contact reviews-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: reviews@aurora.apache.org
Content-Type: multipart/alternative;
 boundary="===============2709236466316378524=="
MIME-Version: 1.0
Subject: Re: Review Request 63199: Refactor staticallyBannedOffers into a LRU
 cache
From: Reza Motamedi <reza.motamedi@gmail.com>
To: Bill Farner <wfarner@apache.org>, David McLaughlin <david@dmclaughlin.com>,
 Stephan Erb <serb@apache.org>,
 Santhosh Kumar Shanmugham <santhoshkumar.s@gmail.com>
Cc: Aurora <reviews@aurora.apache.org>, Reza Motamedi <reza.motamedi@gmail.com>,
 Jordan Ly <jordan.ly8@gmail.com>
Date: Mon, 23 Oct 2017 16:39:30 -0000
Message-ID: <20171023163930.11020.32675@reviews-vm2.apache.org>
Auto-Submitted: auto-generated
Sender: Reza Motamedi <noreply@reviews.apache.org>
References: <20171021055347.61420.80699@reviews-vm2.apache.org>
In-Reply-To: <20171021055347.61420.80699@reviews-vm2.apache.org>
Reply-To: Reza Motamedi <reza.motamedi@gmail.com>
archived-at: Mon, 23 Oct 2017 16:39:35 -0000

--===============2709236466316378524==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63199/#review188953
-----------------------------------------------------------


src/main/java/org/apache/aurora/scheduler/offers/OffersModule.java
Lines 109 (patched)
<https://reviews.apache.org/r/63199/#comment265900>

    Would it be easy to find the "correct" value here? The correct value seems to be correlated to a lot of things. How sensitive is the stablity of clusters to this value?


- Reza Motamedi


On Oct. 21, 2017, 5:53 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63199/
> -----------------------------------------------------------
> 
> (Updated Oct. 21, 2017, 5:53 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Using the new `hold_offers_forever` option, it is possible for the `staticallyBannedOffers` to grow very large in size as we never release offers.
> 
> As an alternative to https://reviews.apache.org/r/63121/, I propose changing `staticallyBannedOffers` into a LRU cache which expires entries after `min_offer_hold_time` + `offer_hold_jitter_window` (referred to as `maxOfferHoldTime`), while also taking an option for a maximum size for the cache. I believe that this approach has a couple of benefits:
> 
> 1. The current behavior of `staticallyBannedOffers` is (kinda) preserved. Entries will no longer be removed when the offer is used, but they will be removed within `maxOfferHoldTime`. This means cluster operators will not have to think about the new `offer_static_ban_cache_max_size` if they aren't affected by the memory leak now.
> 2. Cluster operators that use Aurora as a single framework and hold offers indefinitely can cap the size of the cache to avoid the memory leak.
> 3. Using an LRU cache greatly benefits quickly recurring crons and job updates.
> 
> 
> Diffs
> -----
> 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 5a9099bf9dd292249d72bc3a7604fbb3394f30ea 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 7011a4cc9eea827cdd54698aaed1a653774bce7f 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferSettings.java e060f2073dce4d2486d1ee2bfd873fe75167c6d0 
>   src/main/java/org/apache/aurora/scheduler/offers/OffersModule.java e6b2c55e4f33f9a603157236766425edcaff10e7 
>   src/test/java/org/apache/aurora/scheduler/config/CommandLineTest.java 5b502442163581daa4d7954b09c00bdc3680a726 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 6c8434e9cfe46ef63ff10c6f059ecb99981f29a2 
> 
> 
> Diff: https://reviews.apache.org/r/63199/diff/4/
> 
> 
> Testing
> -------
> 
> Unit tests pass.
> Deployed on a scale test cluster and saw that a) `staticallyBannedOffers` memory leak fixed with correct options and b) lowered assignment time for quickly recurring crons and rescheduled jobs.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>


--===============2709236466316378524==--