Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@zookeeper.apache.org
From: geek101 <git@git.apache.org>
To: dev@zookeeper.apache.org
Reply-To: dev@zookeeper.apache.org
Message-ID: <git-pr-185-zookeeper@git.apache.org>
Subject: [GitHub] zookeeper pull request #185: Branch 3.5 FLE and ZAB SSL socket support [DO N...
Content-Type: text/plain
Date: Tue,  7 Mar 2017 04:32:32 +0000 (UTC)
archived-at: Tue, 07 Mar 2017 04:32:40 -0000

GitHub user geek101 opened a pull request:

    https://github.com/apache/zookeeper/pull/185

    Branch 3.5 FLE and ZAB SSL socket support [DO NOT MERGE]

    # Zookeeper Dynamic Quorum SSL (work in progress)
    
    Provides SSL for Leader Election and ZAB i.e ports 3888 and 2888.
    
    Goal of this patch is to build on top of SSL changes for [branch-3.4](https://github.com/geek101/zookeeper/blob/branch-3.4/README_SSL.md) and in 
    the spirit of branch-3.5 provide support for dynamic reconfiguration, i.e do 
    not violate safety and liveliness even when SSL is enabled.
    
    ### TODO
    
    * CA based cert verification currently has no support for changing CA in a 
    fault-tolerant way. Alternative to changing CA is to revoke the quorum peer 
    to be removed via CRL(s) hence this needs more discussion/debate perhaps. 
    
    * Needs test framework and/or test cases to verify new functionality, this is 
    probably a significant amount of work gating this patch among other things.
    
    ### Self Signed Certs How To
    
    The idea here is to propagate the new member(s) certificate fingerprint(s) 
    via the secure channel to the quorum via the reconfig() API.
    
    Each quorum peer will have its self-signed cert finger print (typically like 
    a SHA-256 digest) embedded into its server string.
    
    ```
    server.1=125.23.63.23:2780:2783:participant:cert:SHA-256-XXXX;2791
    ```
    
    We also have support for both plain and secure port for ZookeeperServer hence 
    the config has been extended to reflect that as follows.
    
    ```
    server.1=125.23.63.23:2780:2783:participant:cert:SHA-256-XXXX;plain:2791;
    secure:2891
    ```
    
    A reconfiguration operation would work by submitting the new server 
    config (or the new quorum list) to reconfig() API.
    
    This has been tested to work with the current state of the patch.
    
    ### Some details
    
    * [X509Util](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/common/X509Util.java) 
    becomes first class citizen and [QuorumX509Util](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/util/QuorumX509Util.java) and [ServerX509Util](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/util/ServerX509Util.java)
    extend it.
    * [ZKConfig](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/common/ZKConfig.java) 
    becomes an abstract class and [QuorumSslConfig](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java) and 
    [ZookeeperServerConfig](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/ZookeeperServerConfig.java) implement it.
    * [QuorumServer](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L278) gets the parsing code for the extra cert information and gets 
    help from new [SSLCertCfg](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/SSLCertCfg.java) class. Dynamic config generation is handled 
    transparently due to this.
    * [ZKDynamicX509TrustManager](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/util/ZKDynamicX509TrustManager.java) handles the dynamic verification of certs and 
    gets help from QuorumPeer's new API, 
    [getQuorumServerFingerPrintByElectionAddress()](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L1642) and 
    [getQuorumServerFingerPrintByCert()](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L1661)
    * Support for a Quorum peer to also be authenticated as a [ZK client](https://github.com/geek101/zookeeper/blob/branch-3.5-ssl-review5/src/java/main/org/apache/zookeeper/server/util/ServerX509Util.java#L62) (this 
    will be removed if it breaks security and or is not needed)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/geek101/zookeeper branch-3.5-ssl-review5

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zookeeper/pull/185.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #185
    
----
commit 9fea6674fbcaead59e090c28dbd7a59e864c442e
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-08-14T02:27:03Z

    SSL support for ZAB and FLE.
    
    Code for dynamic reconfig() and SSL support.

commit b249a1d79d7f46ddba3855922348cc00e2c18ad7
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-08-18T05:04:21Z

    Pass ZKConfig() around to make things work.

commit 2f2c8cbaadf8db9c7aa4b51588d5c35547163990
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-08-21T20:00:45Z

    Fix QuorumPeer constructor call.

commit 98cc6c48e0ead57be221c0bb3729f9f1bde37382
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-08-21T20:43:32Z

    Fix a typo, pass the host ip part of the string.

commit 8e35e86bd0de86b4378bcb8a7e0cb920f263b9a5
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-08-21T23:50:04Z

    Make ZookeeperServer SSL auth to force client to be
    one of the ZK nodes. For now this is ok but this is
    not useful for everyone.
    TODO: Create a different trust manager system property
    for north-south SSL perhaps along with QuorumPeer dynamic
    verification.

commit a040e08cb23ec16456277a6cc3e12566e15b369f
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-09-05T22:03:41Z

    Seperate SSL config for client and quorum
    
    Quorum servers will have different properties for
    SSL config, example:
    -Dquorum.ssl.enabled=true
    -Dquorum.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks
    -Dquorum.ssl.keyStore.password=testpass
    -Dquorum.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks
    -Dquorum.ssl.trustStore.password=testpass
    
    Enable each zookeeper node will be able to also allowed to be authenticated
    as a client using dynamic reconfig.
    
    Basic algorithm for various SSL connections.
    
    Client:
      1. Use the given truststore if available
      2. Use the given server's digest
    
    Server:
      1. Use the given truststore if available
      2. Use dynamic config to check against each server's digest
    
    Quorum:
      1. Use the given truststore if available
      2. Use dyanmic config to check against each server's digest

commit 351de6e5a97980bdba127ba5dbbf265ce8e979d1
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-09-06T06:12:35Z

    Fail verification if Truststore has atleast one cert.
    
    With chained verification a truststore with atleast one cert
    trumps verification down stream.
    
    No more support for "cacert" in server string.

commit c8f16214609948834a9cc087294cabc217124f8b
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-09-06T08:20:22Z

    Self signed certs are not verified via Truststore.
    
    TODO: Fix this later.

commit f736859e06d045bd04aaa9227021d98d938991c5
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-09-14T07:48:30Z

    Server config string has both plain and secure address
    
    Now the client parts of server string could look like this:
    plain:127.0.1.1:2181;secure:127.0.1.1:2281
    
    Its upto caller to ensure secureClientAddress and
    secureClientPortAddress are same in the secure section above.

commit 0b0ce20081832c1630d6da9679cbde81b0ecf413
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2016-09-15T05:17:13Z

    Fix cert parsing a bit.

commit 0ccee0ac33261bc55c49f06cf51cb9a24ea06c63
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2017-03-07T04:19:41Z

    Fix test code due to Quorum Peer constructor change
    and also since JUTE_MAXBUFFER is moved.

commit 30450268282d307e04c684dbcb0abb9c9bc59dca
Author: Powell Molleti <powellm79@yahoo.com>
Date:   2017-03-07T04:25:29Z

    Add README to help explain what this code tries to accomplish.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---