Return-Path: X-Original-To: apmail-aurora-reviews-archive@minotaur.apache.org Delivered-To: apmail-aurora-reviews-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4D17811985 for ; Tue, 24 Jun 2014 23:25:30 +0000 (UTC) Received: (qmail 88749 invoked by uid 500); 24 Jun 2014 23:25:30 -0000 Delivered-To: apmail-aurora-reviews-archive@aurora.apache.org Received: (qmail 88709 invoked by uid 500); 24 Jun 2014 23:25:30 -0000 Mailing-List: contact reviews-help@aurora.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: reviews@aurora.incubator.apache.org Delivered-To: mailing list reviews@aurora.incubator.apache.org Received: (qmail 88697 invoked by uid 99); 24 Jun 2014 23:25:29 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Jun 2014 23:25:29 +0000 X-ASF-Spam-Status: No, hits=-1997.8 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 24 Jun 2014 23:25:29 +0000 Received: (qmail 86115 invoked by uid 99); 24 Jun 2014 23:25:03 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Jun 2014 23:25:03 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 5D6F71DB281; Tue, 24 Jun 2014 23:24:53 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============5582851926567750488==" MIME-Version: 1.0 Subject: Re: Review Request 21440: Implementing parallel updater From: "Maxim Khutornenko" To: "Brian Wickman" , "Mark Chu-Carroll" Cc: "Aurora" , "Maxim Khutornenko" Date: Tue, 24 Jun 2014 23:24:53 -0000 Message-ID: <20140624232453.22595.66305@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Maxim Khutornenko" X-ReviewGroup: Aurora X-ReviewRequest-URL: https://reviews.apache.org/r/21440/ X-Sender: "Maxim Khutornenko" References: <20140606175528.10017.56177@reviews.apache.org> In-Reply-To: <20140606175528.10017.56177@reviews.apache.org> Reply-To: "Maxim Khutornenko" X-ReviewRequest-Repository: aurora X-Virus-Checked: Checked by ClamAV on apache.org --===============5582851926567750488== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21440/ ----------------------------------------------------------- (Updated June 24, 2014, 11:24 p.m.) Review request for Aurora, Mark Chu-Carroll and Brian Wickman. Changes ------- Adding multiplexing for the getTasksStatus calls. Bugs: AURORA-350 https://issues.apache.org/jira/browse/AURORA-350 Repository: aurora Description ------- The updater now spawns upto batch_size threads to process one instance per thread. All mutating calls are multiplexed by the SchedulerMux to do batch kill/add/restart calls. This is the first step towards a fully multiplexed SchedulerProxy and is intended to mitigate LDAP/scheduler load. Diffs (updated) ----- src/main/python/apache/aurora/client/api/BUILD c205a7d5f281c2b9e52d778c3d84647f25dafc15 src/main/python/apache/aurora/client/api/error_handling_thread.py PRE-CREATION src/main/python/apache/aurora/client/api/instance_watcher.py e09aa9a6c32c17f13c9b8ff3a589919587bd839b src/main/python/apache/aurora/client/api/job_monitor.py d176995fca68d42fcc2d3989483eaf520d0d737f src/main/python/apache/aurora/client/api/scheduler_client.py 10a956c50de893c8380ef1f763ed7d6686b5c00c src/main/python/apache/aurora/client/api/scheduler_mux.py PRE-CREATION src/main/python/apache/aurora/client/api/task_util.py PRE-CREATION src/main/python/apache/aurora/client/api/updater.py c592651770f9426fd42b7f92fe2185e1e3065c43 src/main/python/apache/aurora/client/api/updater_util.py 6b689c11b86c99ef5538c0b7bf4dbb81e0df2d13 src/test/python/apache/aurora/client/api/BUILD 804195b719cad7652b3ccb15f03c7c80fc2a13f3 src/test/python/apache/aurora/client/api/test_instance_watcher.py 723a5b6f429f85202ee4ad3a004f019cec632a98 src/test/python/apache/aurora/client/api/test_job_monitor.py 3aa96075e085ac7c2607a8c303ee5daf6e3e43b0 src/test/python/apache/aurora/client/api/test_scheduler_mux.py PRE-CREATION src/test/python/apache/aurora/client/api/test_task_util.py PRE-CREATION src/test/python/apache/aurora/client/api/test_updater.py 48f82c8d9606b7e3bae77eaabbbca8edc92ab045 src/test/python/apache/aurora/client/cli/test_create.py b9313be657ebaf9c79a7695b75abe2d90a7d75dd src/test/python/apache/aurora/client/cli/test_kill.py e9e167991cd06e556dbbc839eb9a808eb7d88fe9 src/test/python/apache/aurora/client/cli/test_restart.py 407eb41eb6e78721ef29baefdee5699e605193f9 src/test/python/apache/aurora/client/cli/test_update.py 44a180e88893fea1d7acf1862d5aaa95d98598f5 src/test/python/apache/aurora/client/cli/util.py a2c9d09c58243250baefab20000ee51f5ea3b411 src/test/python/apache/aurora/client/commands/test_create.py 4cd1343b19e9d13abe20d3b88dc7460ec512d57b src/test/python/apache/aurora/client/commands/test_kill.py 94bbe1bfff1c8d333837aa6bb14700714f46288b src/test/python/apache/aurora/client/commands/test_restart.py 0c6d5a069c43d96098c126aa340a734cb37c667a src/test/python/apache/aurora/client/commands/test_update.py ea8e092267fd65ccd24ca1be3c04f50ba89f5236 src/test/python/apache/aurora/client/commands/util.py b1822f27d2890469efefc5bfa0878f29163c167a src/test/python/apache/aurora/client/fake_scheduler_proxy.py 2a4773c81efb390385f675854e9631500b263a45 src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 80871273fc4d47558253e6b09c92724e8693bc11 src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora fc723cf232ddbc10458fc394e37358c8523118c2 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 9c5652829ac306dda5f7e95e164c85713e18988f src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh 2af256d65850bd861111279dff4b5c53f234cf7a Diff: https://reviews.apache.org/r/21440/diff/ Testing ------- ./pants src/test/python:all src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh Thanks, Maxim Khutornenko --===============5582851926567750488==--