Return-Path: X-Original-To: apmail-directory-dev-archive@www.apache.org Delivered-To: apmail-directory-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 25DC9CD2E for ; Tue, 23 Dec 2014 17:58:27 +0000 (UTC) Received: (qmail 44100 invoked by uid 500); 23 Dec 2014 17:58:27 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 44048 invoked by uid 500); 23 Dec 2014 17:58:27 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 44038 invoked by uid 99); 23 Dec 2014 17:58:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Dec 2014 17:58:26 +0000 X-ASF-Spam-Status: No, hits=-2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of kai.zheng@intel.com designates 134.134.136.24 as permitted sender) Received: from [134.134.136.24] (HELO mga09.intel.com) (134.134.136.24) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Dec 2014 17:58:21 +0000 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP; 23 Dec 2014 09:56:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.07,633,1413270000"; d="scan'208,217";a="628327638" Received: from kmsmsx152.gar.corp.intel.com ([172.21.73.87]) by orsmga001.jf.intel.com with ESMTP; 23 Dec 2014 09:57:59 -0800 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by KMSMSX152.gar.corp.intel.com (172.21.73.87) with Microsoft SMTP Server (TLS) id 14.3.195.1; Wed, 24 Dec 2014 01:57:58 +0800 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.240]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.67]) with mapi id 14.03.0195.001; Wed, 24 Dec 2014 01:57:56 +0800 From: "Zheng, Kai" To: "dev@directory.apache.org" Subject: About Haox project Thread-Topic: About Haox project Thread-Index: AdAeuV7uajnBl1DRSMSDFgr2XzQh6Q== Date: Tue, 23 Dec 2014 17:57:56 +0000 Message-ID: <8D5F7E3237B3ED47B84CF187BB17B66611A338F6@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: multipart/alternative; boundary="_000_8D5F7E3237B3ED47B84CF187BB17B66611A338F6SHSMSX103ccrcor_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_8D5F7E3237B3ED47B84CF187BB17B66611A338F6SHSMSX103ccrcor_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi all, I'm Kai and a guy from Intel Big Data team. I have been working on Apache H= adoop for some while, and used to be an ApacheDS contributor and committed = the feature of CredentialCahce and a simple Kinit tool, guided by Emmanuel = and Kiran. After that when I was going to implement more Kerberos features = I realized there needs important improvement space in the existing fundamen= tal Kerberos codes. As a prototype I initiated Haox project and have imple= mented an ASN1 encoding/decoding framework, complete encryption and checksu= m crypto, Kerberos codec, Keytab and CredentialCache facilities, KrbClient = API and simple KDC server. It's still in its initial phase and far from ide= al. However as strongly suggested by Emmanuel and Kiran, I should collabora= te with ApacheDS community and continue to make the effort with you togethe= r. It's great and now I'm going to contribute the project to ApacheDS. I ha= ve discussed with Emmanuel and Kiran for quite some time about this. Based = on our discussion, we have the following proposal. It's for your review and= consider. Please kindly provide your feedback and advise. Thanks in advanc= e. 1. First all our vision for the long term, we would think it makes great se= nse to establish a first class Kerberos implementation in Java for the Apac= he world targeting today's environments in cloud, Hadoop and mobile. There= fore we could collaborate together in ApacheDS community to build a "Apache= Kerberos" project. That's why we're here. 2. And as the first step to start with, ApacheDS would accept Haox as a sub= -project, renamed as "Apache Kerberos". Then we consolidate existing Kerber= os implementation codes in ApacheDS into the new project; 3. Afterwards we will make further effort to push "Apache Kerberos" out as = a top level project, probably in a year or so. We have discussed other approaches, like using the good stuffs in Haox to r= eplace the corresponding parts in ApacheDS Kerberos, as one may also sugges= t. So let's figure out what might be good parts implemented in Haox. 1. Haox-ASN1. It's a schema/model driven ASN1 parser and codec. Based o= n the framework new Kerberos types can be easily defined following fixed pa= tterns, without having to take care of ASN1 encoding/decoding procedures. W= e see this as very important because Kerberos itself and related extensions= are very ASN1 heavy. Without the basic friendly library support it's hard = for new developers to contribute on new Kerberos evolving features. We test= ed the codec performance and it outperforms a little. Emmanuel has even con= tributed a few optimization improvements. 2. Kerberos crypto. The crypto framework (based on RFC3961) and library= implement all the encryption types and checksum types that MIT Kerberos su= pports, like DES, DES3, AES, RC4, CAMELLIA, with extensive interoperable te= sts covered. It relies on JCE for encryption but not any Kerberos facilitie= s provided in JRE. Potentially we could port it to other non-Oracle Java en= vironment like Android system, assuming we can implement corresponding encr= yption and hash providers which can be pluggable. 3. Kerberos codec. Based on Haox-Asn1, the codec has implemented all th= e necessary Kerberos types defined in Krb5(RFC4210), Preauth/FAST(RFC6113),= and PKINIT(4556). The core part has some test cases covered already. 4. KDC server. The implementation abstracts an KrbIdentityService API a= nd decouples any specific identity backend from the Kerberos server. We can= provide an LDAP based backend, or allow vendor to implement their own back= ed by SQL database, external LDAP server or whatever. For simple unit test = usage, a memory HashMap is all the needs. This allows our embedded KDC real= ly very lightweight avoiding many unnecessary dependencies. 5. Haox-event. It implements a UDP/TCP mixed network event support base= d on Java NIO and reactor pattern according to Kerberos's needs, where UDP = and TCP are both required, and even in IAKERB, a KDC proxy would serve both= TCP server role and UDP/TCP client role. In Haox-event, all these roles ca= n be done in an unified event receiving, processing and replying. Test case= s are given but it definitely need to tune and improve. Again, it doesn't r= ely on any other libraries, thus based on it KrbClient could easily be supp= orted in other Java environment. Keeping our long term goal in mind, we thought above functionalities, prope= rties and considerations may serve better to establish the foundation for "= Apache Kerberos" project, and it would be much easier to consolidate the ex= isting Kerberos implementation into it. So as a first step for the collaboration, how about: 1. Prepare, which involves: 1) Necessary cleanup; 2) All Kerberos related codes will start package names with "org.apache.ker= beros", all the non-Kerberos related codes will be moved into contrib proje= cts or libraries, like Haox-ASN1, Haox-event, Haox-config, which are actual= ly not Kerberos logic specific. 2. Move Haox to be an ApacheDS sub-project: 1) Renaming Haox as "Apache Kerberos"; 2) A separate GIT repo; 3) A separate coding style, subject to be discussed and determined. Then as the 2nd step, consolidate the necessary parts in existing Kerberos = implementation. Below is the initial list and please complement: 1. Apache Directory Server support. We need to create an LdapIdentityBa= ckend using Apache Directory Server and use it by default, embedded in the = KDC server. 2. Installation packaging and service wrapper. Haox hasn't implemented = such but ApacheDS does. 3. Tests. All the Kerberos related tests should be evaluated and ported= to the new codebase if not obsolete. In addition to above, we also need to consider the following aspects, to ma= ke it good enough for potential users. 1. Adding the missing Javadoc. 2. Adding more tests. 3. Adding the documentation. 4. Building the web site With such done, then we might obsolete the AacheDS Kerberos implementation = and might move all the related codes into an history folder for existing us= ers. We would encourage users to use the new project. For the 3rd step, we would continue to implement some essential and advance= d features, which may include: 1. Cross realm support, which means Apache Kerberos can federate with M= IT Kerberos or MS AD; 2. Full functional client tools, like kinit, klist and kadmin, or other= alternatives; 3. Preauth/FAST framework, which allows to support other authentication= mechanisms based on the secure channel protected by an Armor TGT/key; 4. PKINIT, authenticating user using x509 certificate other than plain = password; 5. Token preauth, authenticating user using a JWT token, to support OAu= th. 6. ... Note tasks in step2 and 3 are not necessarily to be done in the order. We'r= e likely to work on them in parallel. We would encourage more developers to contribute to the effort and develop = the community. Without that, we may not be able to make it to have a high q= uality of Kerberos implementation, with all the important Kerberos feature = gaps being filled. Then finally as the last step, in some appropriate time we would push Apach= e Kerberos out to be an Apache top level project. All the points are just suggestions and subject to your discussion. Your fe= edback is most welcome. As widely discussed in a week or so, this proposal = is to be finalized and then re-posted. Then let's collaborate and make it t= ogether. Thanks for your support. Below is Haox project. https://github.com/drankye/haox At last but not least, many thanks to Emmanuel and Kiran. I can't appreciat= e them anymore. Without their encouraging, I would not have the motivation = though it's still far away from the long term goal of "a first class Kerber= os implementation". I do believe the Java world needs such, and the whole A= pache world needs it as well. Thanks for your great questions, suggestions,= complement points to sort this out. Thanks for helping me understand furth= er about the Apache way, particularly regarding how I should proceed and co= llaborate with you guys in the community. Thank you all. Regards, Kai --_000_8D5F7E3237B3ED47B84CF187BB17B66611A338F6SHSMSX103ccrcor_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi all,<= /span>

 

I’m Kai and a= guy from Intel Big Data team. I have been working on Apache Hadoop for som= e while, and used to be an ApacheDS contributor and committed the feature o= f CredentialCahce and a simple Kinit tool, guided by Emmanuel and Kiran. After that when I was going to implement mor= e Kerberos features I realized there needs important improvement space in t= he existing fundamental Kerberos codes. As a prototype I initiated Haox pro= ject  and have implemented an ASN1 encoding/decoding framework, complete encryption and checksum crypto, Kerb= eros codec, Keytab and CredentialCache facilities, KrbClient API and simple= KDC server. It’s still in its initial phase and far from ideal. Howe= ver as strongly suggested by Emmanuel and Kiran, I should collaborate with ApacheDS community and continue to ma= ke the effort with you together. It’s great and now I’m going t= o contribute the project to ApacheDS. I have discussed with Emmanuel and Ki= ran for quite some time about this. Based on our discussion, we have the following proposal. It’s for your review= and consider. Please kindly provide your feedback and advise. Thanks in ad= vance.

 

1. First all our vi= sion for the long term, we would think it makes great sense to establish a = first class Kerberos implementation in Java for the Apache world targeting = today’s environments  in cloud, Hadoop and mobile. Therefore we could collaborate together in ApacheDS community = to build a “Apache Kerberos” project. That’s why we’= ;re here.

 

2. And as the first= step to start with, ApacheDS would accept Haox as a sub-project, renamed a= s “Apache Kerberos”. Then we consolidate existing Kerberos impl= ementation codes in ApacheDS into the new project;

 

3. Afterwards we wi= ll make further effort to push “Apache Kerberos” out as a top l= evel project, probably in a year or so.

 

We have discussed o= ther approaches, like using the good stuffs in Haox to replace the correspo= nding parts in ApacheDS Kerberos, as one may also suggest. So let’s f= igure out what might be good parts implemented in Haox.

1. &= nbsp;   Haox-ASN1. = It’s a schema/model driven ASN1 parser and codec. Based on the framew= ork new Kerberos types can be easily defined following fixed patterns, with= out having to take care of ASN1 encoding/decoding procedures. We see this as very important because Kerberos itself and rela= ted extensions are very ASN1 heavy. Without the basic friendly library supp= ort it’s hard for new developers to contribute on new Kerberos evolvi= ng features. We tested the codec performance and it outperforms a little. Emmanuel has even contributed a few optimizat= ion improvements.

 

2. &= nbsp;   Kerberos cr= ypto. The crypto framework (based on RFC3961) and library implement all the= encryption types and checksum types that MIT Kerberos supports, like DES, = DES3, AES, RC4, CAMELLIA, with extensive interoperable tests covered. It relies on JCE for encryption but not any K= erberos facilities provided in JRE. Potentially we could port it to other n= on-Oracle Java environment like Android system, assuming we can implement c= orresponding encryption and hash providers which can be pluggable.

 

3. &= nbsp;   Kerberos co= dec. Based on Haox-Asn1, the codec has implemented all the necessary Kerber= os types defined in Krb5(RFC4210), Preauth/FAST(RFC6113), and PKINIT(4556).= The core part has some test cases covered already.

 

4. &= nbsp;   KDC server.= The implementation abstracts an KrbIdentityService API and decouples any s= pecific identity backend from the Kerberos server. We can provide an LDAP b= ased backend, or allow vendor to implement their own backed by SQL database, external LDAP server or whatever. For si= mple unit test usage, a memory HashMap is all the needs. This allows our em= bedded KDC really very lightweight avoiding many unnecessary dependencies.

 

5. &= nbsp;   Haox-event.= It implements a UDP/TCP mixed network event support based on Java NIO and = reactor pattern according to Kerberos’s needs, where UDP and TCP are = both required, and even in IAKERB, a KDC proxy would serve both TCP server role and UDP/TCP client role. In Haox-ev= ent, all these roles can be done in an unified event receiving, processing = and replying. Test cases are given but it definitely need to tune and impro= ve. Again, it doesn’t rely on any other libraries, thus based on it KrbClient could easily be supported in o= ther Java environment.

 

Keeping our long te= rm goal in mind, we thought above functionalities, properties and considera= tions may serve better to establish the foundation for “Apache Kerber= os” project, and it would be much easier to consolidate the existing Kerberos implementation into it.

 

So as a first step = for the collaboration, how about:

1. &= nbsp;   Prepare, wh= ich involves:

1) Necessary cleanup;

2) All Kerberos related codes will start package names with &= #8220;org.apache.kerberos”, all the non-Kerberos related codes will b= e moved into contrib projects or libraries, like Haox-ASN1, Haox-event, Haox-config, which are actually not Kerberos logic specific. <= o:p>

 

2. Move Haox to be = an ApacheDS sub-project:

1) Renaming Haox as= “Apache Kerberos”;

2) A separate GIT r= epo;

3) A separate codin= g style, subject to be discussed and determined.

 

Then as the 2nd ste= p, consolidate the necessary parts in existing Kerberos implementation. Bel= ow is the initial list and please complement:

1. &= nbsp;   Apache Dire= ctory Server support. We need to create an LdapIdentityBackend using Apache= Directory Server and use it by default, embedded in the KDC server.

2. &= nbsp;   Installatio= n packaging and service wrapper. Haox hasn’t implemented such but Apa= cheDS does.

3. &= nbsp;   Tests. All = the Kerberos related tests should be evaluated and ported to the new codeba= se if not obsolete.

 

In addition to abov= e, we also need to consider the following aspects, to make it good enough f= or potential users.

1. &= nbsp;   Adding the = missing Javadoc.

2. &= nbsp;   Adding more= tests.

3. &= nbsp;   Adding the = documentation.

4. Building the web= site

 

With such done, the= n we might obsolete the AacheDS Kerberos implementation and might move all = the related codes into an history folder for existing users. We would encou= rage users to use the new project.

<= /p>

For the 3rd step, we would continue to implement some essential and advanced featur= es, which may include:

1. &= nbsp;   Cross realm= support, which means Apache Kerberos can federate with MIT Kerberos or MS = AD;

2. &= nbsp;   Full functi= onal client tools, like kinit, klist and kadmin, or other alternatives;

3. &= nbsp;   Preauth/FAS= T framework, which allows to support other authentication mechanisms based = on the secure channel protected by an Armor TGT/key;

4. &= nbsp;   PKINIT, aut= henticating user using x509 certificate other than plain password;

5. &= nbsp;   Token preau= th, authenticating user using a JWT token, to support OAuth.

6. &= nbsp;   …

 

Note tasks in step2= and 3 are not necessarily to be done in the order. We’re likely to w= ork on them in parallel.

 

We would encourage = more developers to contribute to the effort and develop the community. With= out that, we may not be able to make it to have a high quality of Kerberos = implementation, with all the important Kerberos feature gaps being filled.

 

Then finally as the= last step, in some appropriate time we would push Apache Kerberos out to b= e an Apache top level project.

 

All the points are = just suggestions and subject to your discussion. Your feedback is most welc= ome. As widely discussed in a week or so, this proposal is to be finalized = and then re-posted. Then let’s collaborate and make it together.

 

Thanks for your sup= port. Below is Haox project.

https://github.com/drankye/haox

 

At last but not lea= st, many thanks to Emmanuel and Kiran. I can’t appreciate them anymor= e. Without their encouraging, I would not have the motivation though itR= 17;s still far away from the long term goal of “a first class Kerberos implementation”. I do believe the Java world ne= eds such, and the whole Apache world needs it as well. Thanks for your grea= t questions, suggestions, complement points to sort this out. Thanks for he= lping me understand further about the Apache way, particularly regarding how I should proceed and collaborate wi= th you guys in the community.

 

Thank you all.=

 

Regards,=

Kai &nb= sp;     

--_000_8D5F7E3237B3ED47B84CF187BB17B66611A338F6SHSMSX103ccrcor_--