commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gangumalla, Uma" <uma.ganguma...@intel.com>
Subject Chimera as new component in Apache Commons
Date Tue, 16 Feb 2016 21:36:32 GMT
Hi Devs,



Recently we worked with Spark community to implement the shuffle encryption. While implementing
that, we realized some/most of the code in Apache Hadoop encryption code and this implementation
code have to be duplicated. This leads to an idea to create separate reusable library, named
it as Chimera (https://github.com/intel-hadoop/chimera). It is an optimized cryptographic
library. It provides Java API for both cipher level and Java stream level to help developers
implement high performance AES encryption/decryption with the minimum code and effort.



We know that Java has Cipher implementations, but why we need this optimized cryptographic
library:

1. Performance is very critical for encryption and decryption. The JDK Cipher implementation
of AES are not yet optimized with the modern hardware. For example, the optimized implementation
is 17x+ faster than JDK6 implementations for some modes such as CBC decryption, CTR and GCM.
Even some optimizations has included JDK7 or JDK8, there are still 5x to 6x gap with the most
optimized implementations.

2. Java Stream based API on cryptographic data stream. Cipher API is powerful but a lot of
code needs to be written for layered stream processing applications. The design pattern is
very common in modern applications such as Hadoop or Spark.



Chimera was originally based Hadoop crypto code but was improved and generalized a lot for
supporting wider scope of data encryption needs for more components in the community. The
encryption related code in Hadoop was developed a year and so far it is running well. So we
feel that code part of stable enough already.



So, we propose to contribute this Chimera (optimized encryption library) code to Apache Commons
and we wanted to have independent release cycles for this module like any other modules in
Apache Commons. This module is basically provides Java based interfaces for encryption based
IO and It will have native based AES-NI encryption integration code.



We already discussed about this proposal in Apache Hadoop dev lists and the discussion conclusion
was positive to contribute this module to Apache Commons.



We need your help and support in adopting this code to make as Apache Commons sub module and
establish for making it to have its own development community (of course we can discuss more
about this factors in this thread). And Hadoop and Spark will be the two visible projects
to use it. We do expect there will be more projects using it.



Once Apache Commons PMC agreed to place this module under Commons, I will work on getting
the interested developers etc for establishing Chimera development community as part of next
steps. Please help on the process.



Regards,

Uma (An Apache Hadoop PMC member)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message