aurora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s...@apache.org
Subject aurora git commit: Make announced scheduler endpoint name configurable.
Date Wed, 18 Jan 2017 09:26:32 GMT
Repository: aurora
Updated Branches:
  refs/heads/master c223e666f -> 6ad4c8728


Make announced scheduler endpoint name configurable.

We decided to co-deploy an HTTPS enabled reverse proxy in front of each of our
Aurora schedulers. The proxy instances bind to `public_ip:8081` and the
schedulers to `localhost:8081`. By announcing the scheduler endpoint as `https`
we can ensure the default Aurora [client connects via HTTPS](https://github.com/apache/aurora/blob/master/src/main/python/apache/aurora/client/api/scheduler_client.py#L176-L178).

Default:

    [zk: 127.0.0.1:2181(CONNECTED) 5] get /aurora/scheduler/member_0000000011
    {"serviceEndpoint":{"host":"aurora.local","port":8081},"additionalEndpoints":{"http":{"host":"aurora.local","port":8081}},"status":"ALIVE"}

When running with `-serverset_endpoint_name=https`:

    [zk: 127.0.0.1:2181(CONNECTED) 0] get /aurora/scheduler/member_0000000019
    {"serviceEndpoint":{"host":"aurora.local","port":8081},"additionalEndpoints":{"https":{"host":"aurora.local","port":8081}},"status":"ALIVE"}

Bugs closed: AURORA-343

Reviewed at https://reviews.apache.org/r/55583/


Project: http://git-wip-us.apache.org/repos/asf/aurora/repo
Commit: http://git-wip-us.apache.org/repos/asf/aurora/commit/6ad4c872
Tree: http://git-wip-us.apache.org/repos/asf/aurora/tree/6ad4c872
Diff: http://git-wip-us.apache.org/repos/asf/aurora/diff/6ad4c872

Branch: refs/heads/master
Commit: 6ad4c8728b8f024a04a16be52a53ba96cc185ca3
Parents: c223e66
Author: Stephan Erb <serb@apache.org>
Authored: Wed Jan 18 10:25:54 2017 +0100
Committer: Stephan Erb <serb@apache.org>
Committed: Wed Jan 18 10:25:54 2017 +0100

----------------------------------------------------------------------
 RELEASE-NOTES.md                                |  3 ++
 docs/operations/security.md                     | 50 ++++++++++++++------
 docs/reference/scheduler-configuration.md       |  6 +++
 .../aurora/scheduler/app/SchedulerMain.java     |  6 ++-
 4 files changed, 50 insertions(+), 15 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/aurora/blob/6ad4c872/RELEASE-NOTES.md
----------------------------------------------------------------------
diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md
index 164d7a3..7d01c90 100644
--- a/RELEASE-NOTES.md
+++ b/RELEASE-NOTES.md
@@ -28,6 +28,9 @@
   util `min_consecutive_successes` consecutive health checks have passed.
 - The default logging output has been changed to remove line numbers and inner class information
in
   exchange for faster logging.
+- Support the deployment of the Aurora scheduler behind HTTPS-enabled reverse proxies: By
launching
+  scheduler via `-serverset_endpoint_name=https` you can ensure the Aurora client will correctly
+  discover HTTPS support via the ZooKeeper-based discovery mechanism.
 
 ### Deprecations and removals:
 

http://git-wip-us.apache.org/repos/asf/aurora/blob/6ad4c872/docs/operations/security.md
----------------------------------------------------------------------
diff --git a/docs/operations/security.md b/docs/operations/security.md
index 46e0b8a..2bb7046 100644
--- a/docs/operations/security.md
+++ b/docs/operations/security.md
@@ -21,10 +21,11 @@ controls for talking to ZooKeeper.
 		- [Caveats](#caveats)
 - [Implementing a Custom Realm](#implementing-a-custom-realm)
 	- [Packaging a realm module](#packaging-a-realm-module)
-- [Known Issues](#known-issues)
 - [Announcer Authentication](#announcer-authentication)
     - [ZooKeeper authentication configuration](#zookeeper-authentication-configuration)
     - [Executor settings](#executor-settings)
+- [Scheduler HTTPS](#scheduler-https)
+- [Known Issues](#known-issues)
 
 # Enabling Security
 
@@ -275,18 +276,6 @@ class name:
 -shiro_realm_modules=KERBEROS5_AUTHN,INI_AUTHNZ,com.example.MyRealmModule
 ```
 
-# Known Issues
-
-While the APIs and SPIs we ship with are stable as of 0.8.0, we are aware of several incremental
-improvements. Please follow, vote, or send patches.
-
-Relevant tickets:
-* [AURORA-343](https://issues.apache.org/jira/browse/AURORA-343): HTTPS support
-* [AURORA-1248](https://issues.apache.org/jira/browse/AURORA-1248): Client retries 4xx errors
-* [AURORA-1279](https://issues.apache.org/jira/browse/AURORA-1279): Remove kerberos-specific
build targets
-* [AURORA-1293](https://issues.apache.org/jira/browse/AURORA-1291): Consider defining a JSON
format in place of INI
-* [AURORA-1179](https://issues.apache.org/jira/browse/AURORA-1179): Supported hashed passwords
in security.ini
-* [AURORA-1295](https://issues.apache.org/jira/browse/AURORA-1295): Support security for
the ReadOnlyScheduler service
 
 # Announcer Authentication
 The Thermos executor can be configured to authenticate with ZooKeeper and include
@@ -337,4 +326,37 @@ All properties of the `permissions` object will default to False if not
provided
 
 ## Executor settings
 To enable the executor to authenticate against ZK, `--announcer-zookeeper-auth-config` should
be
-set to the configuration file.
\ No newline at end of file
+set to the configuration file.
+
+
+# Scheduler HTTPS
+
+The Aurora scheduler does not provide native HTTPS support ([AURORA-343](https://issues.apache.org/jira/browse/AURORA-343)).
+It is therefore recommended to deploy it behind an HTTPS capable reverse proxy such as nginx
or Apache2.
+
+A simple setup is to launch both the reverse proxy and the Aurora scheduler on the same port,
but
+bind the reverse proxy to the public IP of the host and the scheduler to localhost:
+
+    -ip=127.0.0.1
+    -http_port=8081
+
+If your clients connect to the scheduler via [`proxy_url`](../reference/scheduler-configuration.md),
+you can update it to `https`. If you use the ZooKeeper based discovery instead, the scheduler
+needs to be launched via
+
+    -serverset_endpoint_name=https
+
+in order to announce its HTTPS support within ZooKeeper.
+
+
+# Known Issues
+
+While the APIs and SPIs we ship with are stable as of 0.8.0, we are aware of several incremental
+improvements. Please follow, vote, or send patches.
+
+Relevant tickets:
+* [AURORA-1248](https://issues.apache.org/jira/browse/AURORA-1248): Client retries 4xx errors
+* [AURORA-1279](https://issues.apache.org/jira/browse/AURORA-1279): Remove kerberos-specific
build targets
+* [AURORA-1293](https://issues.apache.org/jira/browse/AURORA-1291): Consider defining a JSON
format in place of INI
+* [AURORA-1179](https://issues.apache.org/jira/browse/AURORA-1179): Supported hashed passwords
in security.ini
+* [AURORA-1295](https://issues.apache.org/jira/browse/AURORA-1295): Support security for
the ReadOnlyScheduler service

http://git-wip-us.apache.org/repos/asf/aurora/blob/6ad4c872/docs/reference/scheduler-configuration.md
----------------------------------------------------------------------
diff --git a/docs/reference/scheduler-configuration.md b/docs/reference/scheduler-configuration.md
index d4e0a9a..3e3d799 100644
--- a/docs/reference/scheduler-configuration.md
+++ b/docs/reference/scheduler-configuration.md
@@ -84,6 +84,8 @@ Optional flags:
 	Specifies the frequency at which snapshots of local storage are taken and written to the
log.
 -enable_cors_for
 	List of domains for which CORS support should be enabled.
+-enable_db_metrics (default true)
+	Whether to use MyBatis interceptor to measure the timing of intercepted Statements.
 -enable_h2_console (default false)
 	Enable H2 DB management console.
 -enable_mesos_fetcher (default false)
@@ -210,6 +212,8 @@ Optional flags:
 	If false, Docker tasks may run without an executor (EXPERIMENTAL)
 -scheduling_max_batch_size (default 3) [must be > 0]
 	The maximum number of scheduling attempts that can be processed in a batch.
+-serverset_endpoint_name (default http)
+	Name of the scheduler endpoint published in ZooKeeper.
 -shiro_ini_path
 	Path to shiro.ini for authentication and authorization configuration.
 -shiro_realm_modules (default [class org.apache.aurora.scheduler.http.api.security.IniShiroRealmModule])
@@ -224,6 +228,8 @@ Optional flags:
 	Log all queries that take at least this long to execute.
 -slow_query_log_threshold (default (25, ms))
 	Log all queries that take at least this long to execute.
+-snapshot_hydrate_stores (default [locks, hosts, quota, job_updates])
+	Which H2-backed stores to fully hydrate on the Snapshot.
 -stat_retention_period (default (1, hrs))
 	Time for a stat to be retained in memory before expiring.
 -stat_sampling_interval (default (1, secs))

http://git-wip-us.apache.org/repos/asf/aurora/blob/6ad4c872/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
----------------------------------------------------------------------
diff --git a/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java b/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
index 43cc5b4..805e9de 100644
--- a/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
+++ b/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
@@ -82,6 +82,10 @@ public class SchedulerMain {
   @CmdLine(name = "serverset_path", help = "ZooKeeper ServerSet path to register at.")
   private static final Arg<String> SERVERSET_PATH = Arg.create();
 
+  @CmdLine(name = "serverset_endpoint_name",
+      help = "Name of the scheduler endpoint published in ZooKeeper.")
+  private static final Arg<String> SERVERSET_ENDPOINT_NAME = Arg.create("http");
+
   // TODO(Suman Karumuri): Rename viz_job_url_prefix to stats_job_url_prefix for consistency.
   @CmdLine(name = "viz_job_url_prefix", help = "URL prefix for job container stats.")
   private static final Arg<String> STATS_URL_PREFIX = Arg.create("");
@@ -120,7 +124,7 @@ public class SchedulerMain {
     try {
       schedulerService.lead(
           httpSocketAddress,
-          ImmutableMap.of("http", httpSocketAddress),
+          ImmutableMap.of(SERVERSET_ENDPOINT_NAME.get(), httpSocketAddress),
           leaderListener);
     } catch (SingletonService.LeadException e) {
       throw new IllegalStateException("Failed to lead service.", e);


Mime
View raw message