commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen, Haifeng" <haifeng.c...@intel.com>
Subject RE: [crypto][chimera] Next steps
Date Wed, 24 Feb 2016 02:41:15 GMT
Thanks Bourg for these questions.

>> Sorry if it sounds naive, but why not accessing the OpenSSL functions through a JCE
provider instead of building an abstraction layer on top of another abstraction layer (JCE).
The Apache JuiCE project was an attempt to implement this idea a few years ago [1]. With an
OpenSSL based JCE provider the CryptoInput/OutputStream could probably be replaced by javax.crypto.CipherInput/OutputStream.
As for the secure random part the standard java.security.SecureRandomSpi mechanism could be
used to provide an OpenSSL based implementation.
>> What do you think?
This is not naïve and very valid questions. We actually struggled a long time on these points
from Bourg. We in Intel was driving the community to get the full use of hardware acceleration
in encryption for a long time. We tried that way before. Diceros (https://github.com/intel-hadoop/diceros)
project is exactly like what you mentioned. It provides OpenSSL based JCE provider as well
as Secure Random SPI.

Technically and architecturally, this approach is nicely fit what the JCE architecture targets
to do. But a JCE Provider needs to be officially certificated and signed, which can only be
built and distributed by the specific organization such as Intel. This will prevent its adoption
in the open source community who put much their trust on Apache open source. The developer
cannot build their own version unless they have Oracle certificated private key. Another issue
of this approach is the need for user to configure their JRE environment with the new JCE
provider through java.security file (Although this can be overcome technically by dynamic
register).

While Chimera approach is similar to JCE approach technically but instead as a JCD provider,
it acts as a "reference it and use" library. 

This second value proposition for Chimera is for its stream layer and its abstraction of Input
and Output interface. Bourg mentioned javax CipherInput and CipherOutputStream. The layer
will support Stream, Channel input out of box and can be easily extended with new Input and
Output sources. We also see the importance of CTR mode which provides advanced characteristics
such as random seek features on crypto streams. This is very important for big data encryption
such HDFS which needs to read at the random positions. In this layer, other valuable things
can be added when needed. The utility library is needed in these area in Apache Commons whose
development, enhancement and release will be much agile than JDK improve its classes.

>> Another dumb question, isn't AES-NI enabled by default in Java 8 nowadays [2]? Do
you still get a significant speed up with Chimera in this case? If so I think contributing
the improvements to OpenJDK would be a good idea too, this will benefit everyone in the next
Java releases.
This is a very good question. JDK 6 is not optimized. JDK 7 and JDK 8 is partial optimized
for some modes. We can see 17x throughput improvement for AES/CTR mode. 5x - 6x speedup for
GCM mode. 

>> If so I think contributing the improvements to OpenJDK would be a good idea too,
this will benefit everyone in the next Java releases.
Exactly, we have been working with Oracle team to bring these optimizations in latest JDK
development. In Jan 2016, we committed the optimization on CTR to OpenJDK. (https://bugs.openjdk.java.net/browse/JDK-8143925)


Hope this information helps. Thanks Bourg for these insightful questions.

Thanks,
Haifeng

-----Original Message-----
From: Emmanuel Bourg [mailto:emmanuel.bourg@gmail.com] On Behalf Of Emmanuel Bourg
Sent: Tuesday, February 23, 2016 6:53 PM
To: Commons Developers List <dev@commons.apache.org>
Cc: common-dev@hadoop.apache.org
Subject: Re: [crypto][chimera] Next steps

Hi all,

I got a quick look at the Chimera code. If I understand well it consists in:
- a native interface to the OpenSSL AES & secure random functions
- an abstraction layer to use the JCE or OpenSSL AES implementation
- an abstraction layer to use the JCE or OpenSSL secure random
- encrypting/decrypting input/output streams

Sorry if it sounds naive, but why not accessing the OpenSSL functions through a JCE provider
instead of building an abstraction layer on top of another abstraction layer (JCE). The Apache
JuiCE project was an attempt to implement this idea a few years ago [1]. With an OpenSSL based
JCE provider the CryptoInput/OutputStream could probably be replaced by javax.crypto.CipherInput/OutputStream.
As for the secure random part the standard java.security.SecureRandomSpi mechanism could be
used to provide an OpenSSL based implementation.

What do you think?

Another dumb question, isn't AES-NI enabled by default in Java 8 nowadays [2]? Do you still
get a significant speed up with Chimera in this case? If so I think contributing the improvements
to OpenJDK would be a good idea too, this will benefit everyone in the next Java releases.

Emmanuel Bourg

[1] http://incubator.apache.org/projects/juice.html
[2] http://openjdk.java.net/jeps/164


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Mime
View raw message