Return-Path: X-Original-To: apmail-aurora-reviews-archive@minotaur.apache.org Delivered-To: apmail-aurora-reviews-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6257617AB3 for ; Fri, 7 Nov 2014 23:05:59 +0000 (UTC) Received: (qmail 75622 invoked by uid 500); 7 Nov 2014 23:05:54 -0000 Delivered-To: apmail-aurora-reviews-archive@aurora.apache.org Received: (qmail 75578 invoked by uid 500); 7 Nov 2014 23:05:54 -0000 Mailing-List: contact reviews-help@aurora.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: reviews@aurora.incubator.apache.org Delivered-To: mailing list reviews@aurora.incubator.apache.org Received: (qmail 75567 invoked by uid 99); 7 Nov 2014 23:05:53 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Nov 2014 23:05:53 +0000 X-ASF-Spam-Status: No, hits=-1998.4 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 07 Nov 2014 23:05:51 +0000 Received: (qmail 75165 invoked by uid 99); 7 Nov 2014 23:05:31 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Nov 2014 23:05:31 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 6DE7C1DFD20; Fri, 7 Nov 2014 23:05:35 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============5803932835054793558==" MIME-Version: 1.0 Subject: Re: Review Request 27746: Simplify management of the driver lifecycle using AbstractidleService. From: "Bill Farner" To: "Maxim Khutornenko" , "Kevin Sweeney" Cc: "Bill Farner" , "Aurora ReviewBot" , "Aurora" Date: Fri, 07 Nov 2014 23:05:35 -0000 Message-ID: <20141107230535.10454.84564@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Bill Farner" X-ReviewGroup: Aurora X-ReviewRequest-URL: https://reviews.apache.org/r/27746/ X-Sender: "Bill Farner" References: <20141107225934.10454.82914@reviews.apache.org> In-Reply-To: <20141107225934.10454.82914@reviews.apache.org> Reply-To: "Bill Farner" X-ReviewRequest-Repository: aurora X-Virus-Checked: Checked by ClamAV on apache.org --===============5803932835054793558== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27746/ ----------------------------------------------------------- (Updated Nov. 7, 2014, 11:05 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-920 https://issues.apache.org/jira/browse/AURORA-920 Repository: aurora Description ------- Apologies for the large diff - this is a thread that began with fixing a SchedulerDriver-related race that would exist if we moved to an async event bus. Most of the green code is not new, but extracted from broken-apart and/or deleted files. However, i think this refactor makes the lifecycle of the scheduler (and driver) easier to reason about. Changes of note: - We now use AbstractIdleService to manage the call order on the SchedulerDriver (as well as HistoryPruner, but that is only loosely related). - Many mesos-specific classes were moved to `org.apache.aurora.scheduler.mesos` - Command line args used directly from `DriverFactoryImpl` are now properly injected via `DriverSettings` - Untestable code around creating the driver is cordoned off to `DriverFactoryImpl`, which is a minimal shim over the `MesosSchedulerDriver` constructor - Extracted modules that should not be used in tests: `LibMesosLoadingModule` and `CommandLineDriverSettingsModule` Diffs (updated) ----- config/legacy_untested_classes.txt f1d19210a722ee2c1838c4ffa0eec1f42d05c2d3 src/main/java/org/apache/aurora/scheduler/Driver.java b07378f3772136177c81a65497152e9d6524b0a7 src/main/java/org/apache/aurora/scheduler/DriverFactory.java 9cc04a84a37374ffca418e2ff767992ee23b9f3e src/main/java/org/apache/aurora/scheduler/MesosSchedulerImpl.java c424ecdaefb1b872adb4ef32a2aca088cee26f7f src/main/java/org/apache/aurora/scheduler/MesosTaskFactory.java ad07bcaa3a372efab1b6ba6165f5c44acf8de4cf src/main/java/org/apache/aurora/scheduler/SchedulerLifecycle.java 453f22a8e97dbf535dd99598332e71e944ebf8d2 src/main/java/org/apache/aurora/scheduler/SchedulerModule.java 8d306409bc382fbe5c53bb394d03c923b8f75889 src/main/java/org/apache/aurora/scheduler/app/AppModule.java 9ec3f412a9a9210279a2ce666bd99f61481ab92d src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 048740953629a950c136179c9b880b8dce8fa932 src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 9bc42d3761d3fc72f009716cf087d4ac3c26dce4 src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java feb70a9351fe804b6745304e4cbbf6d010c56579 src/main/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPruner.java 0b023a25292907b8211274f1533d89de7c4b7ce4 src/main/java/org/apache/aurora/scheduler/async/KillRetry.java e656c7ec7349b54b07f8c734266445d1fc15fabb src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 1a45d08c8854a335161476c8321c751f763dc12e src/main/java/org/apache/aurora/scheduler/async/TaskTimeout.java 89140223f4f3acd02ade6fb95734744ef19d89bc src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/DriverFactory.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/DriverFactoryImpl.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/DriverSettings.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/LibMesosLoadingModule.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverModule.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 6d2ac494030018b100c1d01ce373c4896f2ed7d2 src/main/java/org/apache/aurora/scheduler/state/StateModule.java fe16d3a367b4b99b32a25d5ae55a3817a290e5e0 src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 78a9670573261b2bdae0810c104ca7c1e4736ea8 src/main/java/org/apache/aurora/scheduler/storage/log/EntrySerializer.java cbb711fa555c94626963b4d9540a71e6a737abd8 src/main/java/org/apache/aurora/scheduler/storage/log/StreamManagerImpl.java 855573efea68ba4f29b3c7fbfb8da56bbdad352b src/test/java/org/apache/aurora/scheduler/DriverFactoryImplTest.java 0f6192269cc122580869ae7717acad161fac6f1d src/test/java/org/apache/aurora/scheduler/DriverTest.java a96dd87421c32e4da4da4c953dda4c6be4e8e592 src/test/java/org/apache/aurora/scheduler/MesosSchedulerImplTest.java 8dd908e69c71e96d03806e0c4ceb00dab81d756f src/test/java/org/apache/aurora/scheduler/MesosTaskFactoryImplTest.java aa026cf3b6ace5e0aaf9b239aff0c2f3210a0883 src/test/java/org/apache/aurora/scheduler/SchedulerLifecycleTest.java 986160133a3471fc36cad2615c2aa1a7d8983b5a src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 713797182e22956640eeffb796e0a0ff7ec3ddc9 src/test/java/org/apache/aurora/scheduler/app/local/FakeMaster.java 98718c4c75c2bf5b4778b634f140eaa108eaf3b6 src/test/java/org/apache/aurora/scheduler/app/local/LocalSchedulerMain.java 43d73b1895bb0b539196c75344bec1c090fc6410 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 62d8aab3edb03c79777ed3fa1e38e92fd8a06064 src/test/java/org/apache/aurora/scheduler/async/GcExecutorLauncherTest.java 059a276b21bfc271ae45f923190a8eafbd98d723 src/test/java/org/apache/aurora/scheduler/async/KillRetryTest.java 8ad9f5c65ede1bacfdb0eec46aee9af00e625508 src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 15fb7ff617bdd25583c369385193fdcbbf5d8b36 src/test/java/org/apache/aurora/scheduler/async/TaskGroupsTest.java 1d84496e3213510668a82fc1d9a9734617c7eb3d src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 1caaf1487f1fdcdea6feff7d194fa6aae47078e3 src/test/java/org/apache/aurora/scheduler/async/TaskTimeoutTest.java e79327c3b8ead01495e063e5c0e9270568e16f69 src/test/java/org/apache/aurora/scheduler/mesos/SchedulerDriverServiceTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 2fe257416e8fc11be3ebf0dabbf4f63628b3c670 src/test/java/org/apache/aurora/scheduler/sla/MetricCalculatorTest.java fa6d57404bb7905cb23e220edc8f1d59d47e6315 src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java d8a61458203ac696cb3c5ec61b0b4be6320523ca src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java c7da5abc7457719e98c03e1217f99444cb29f4cb src/test/java/org/apache/aurora/scheduler/updater/JobUpdaterIT.java 88319ea869589323c4662d4f62b46b2bc964268b Diff: https://reviews.apache.org/r/27746/diff/ Testing ------- ./gradlew build -Pq ./gradlew run ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh Thanks, Bill Farner --===============5803932835054793558==--