Mailing-List: contact reviews-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: reviews@aurora.apache.org
Content-Type: multipart/alternative;
 boundary="===============0184996032190707798=="
MIME-Version: 1.0
Subject: Re: Review Request 62423: Improve in-process test ZooKeeper support
From: Keisuke Nishimoto <keisuke.nishimoto@gmail.com>
To: Bill Farner <wfarner@apache.org>, Aurora ReviewBot <wfarner@apache.org>,
 Keisuke Nishimoto <keisuke.nishimoto@gmail.com>,
 Aurora <reviews@aurora.apache.org>
Date: Wed, 20 Sep 2017 17:47:52 -0000
Message-ID: <20170920174752.1946.68642@reviews-vm2.apache.org>
Auto-Submitted: auto-generated
Sender: Keisuke Nishimoto <noreply@reviews.apache.org>
References: <20170920165540.1946.57156@reviews-vm2.apache.org>
In-Reply-To: <20170920165540.1946.57156@reviews-vm2.apache.org>
Reply-To: Keisuke Nishimoto <keisuke.nishimoto@gmail.com>
archived-at: Wed, 20 Sep 2017 17:47:57 -0000

--===============0184996032190707798==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On Sept. 20, 2017, 4:55 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLogStreamModule.java
> > Line 148 (original), 150 (patched)
> > <https://reviews.apache.org/r/62423/diff/1/?file=1830036#file1830036line150>
> >
> >     This seems to suggest that `zkClientConfig.getServers()` may return the wrong addresses.  Why is that?

This is because zkClientConfig.getServers() here returns values as specified by -zk_endpoints, which is supposed to be ignored when -zk_in_proc=true.  Since ZooKeeperConfig is not injected and we don't know the port of ZooKeeperTestServer when FlaggedZooKeeperConfig.create() is called, it is not trivial to let ZooKeeperConfig#getServer() return a correct value when in-process ZooKeeper server is used.  I may make ZooKeeperConfig#getServers() non-public instead.


- Keisuke


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62423/#review185801
-----------------------------------------------------------


On Sept. 19, 2017, 10:49 p.m., Keisuke Nishimoto wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62423/
> -----------------------------------------------------------
> 
> (Updated Sept. 19, 2017, 10:49 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Bugs: AURORA-1947
>     https://issues.apache.org/jira/browse/AURORA-1947
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> MesosLogStreamModule tries to connect to ZooKeeper servers specified by -zk_endpoints even when -zk_in_proc=true.  I updated the module to use injected server endpoints which will be based on the ephemeral port assigned to ZooKeeperTestServer if -zk_in_proc=true.  This required to make @ServiceDiscoveryBindings.ZooKeeper public.
> 
> I also tweaked shutdown process of ServiceDiscoveryModule.TestServerService so that it won't close existing ZooKeeper connections before clients close the session.  While just delaying the execution by 1 second doesn't really guarantee that behavior, in practice this achieved clean shutdown of the scheduler with in-process ZooKeeper server.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/discovery/ServiceDiscoveryBindings.java 28cdc4b3b454b3d25008a21c6b12634173e1f878 
>   src/main/java/org/apache/aurora/scheduler/discovery/ServiceDiscoveryModule.java c105dbdbe8339db7f5cca2e5e391fffb4cd87b07 
>   src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLogStreamModule.java 6704a328a4023a178ed8f86ae4772cb04eb2fa8e 
> 
> 
> Diff: https://reviews.apache.org/r/62423/diff/1/
> 
> 
> Testing
> -------
> 
> 1. Launch Mesos master and slave on my laptop.
> 2. Launch Aurora scheduler with following arguments:
> ```
> -backup_dir=/var/lib/aurora/backups
> -cluster_name=local
> -mesos_master_address=localhost:5050
> -serverset_path=/aurora/scheduler
> -ip=127.0.0.1
> -hostname=localhost
> -http_port=8081
> -zk_in_proc=true
> -zk_endpoints=localhost:2181
> -native_log_zk_group_path=/aurora/replicated-log
> -native_log_file_path=/var/db/aurora
> ```
> 3. Observe that there are no ZooKeeper error log outputs caused by missing endpoint.
> 4. Create a simple job, observer it launches normally and then kill it.
> 5. Stop the scheduler by sending /quitquitquit.
> 6. Observe that scheduler process shuts down normally.
> 
> 
> Thanks,
> 
> Keisuke Nishimoto
> 
>


--===============0184996032190707798==--