hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10734) Implementation of true secure random with high performance using hardware random number generator.
Date Thu, 03 Jul 2014 14:26:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051500#comment-14051500
] 

Yi Liu commented on HADOOP-10734:
---------------------------------

[~cmccabe], thanks for the review :-). 

{quote}
I actually have the same problem with the scheme here: JNI calls are expensive... do we know
how many random bits the API user is getting at a time? If that number is small, we might
want to implement batching. 
{quote}
In most cases, we use it to generate key(16bytes, 32bytes, 128bytes, 256bytes), IV(16 bytes),
long (8 bytes).
Furthermore, to make the random bytes good enough, we can’t avoid JNI, even {{java.security.SecureRandom}}
also uses JNI.

{quote}
I also think we should consider using ByteBuffer rather than byte[] array, if performance
is the primary goal.
{quote}
I suppose you mean direct ByteBuffer. Per my understanding, merit of direct ByteBuffer is
to avoid bytes copy.  But {{SecureRandom#nextBytes}} will accept an pre-allocated byte[] array,
if we use direct ByteBuffer for JNI, then there is additional copy in java layer, so the performance
is the same, and we need to manage the direct ByteBuffer.

{quote}
{code}
+  final protected int next(int numBits) {
{code}
Should be private
{quote}
OK, I will update it.

{quote}
{code}
+  public long nextLong() {
+    return ((long)(next(32)) << 32) + next(32);
+  }
{code}
Why use addition rather than bitwise OR here?
{quote}
Bitwise OR is also OK. Actually {{nextLong}}, {{nextFloat}} and {{nextDouble}} are copied
from implementations in {{java.security.SecureRandom}}

{quote}
This is not correct. The type of {{pthread_t}} is not known. If you want a numeric thread
ID, you could try gettid on Linux.
{quote}
Can you explain a bit more, I’m not sure I get your meaning. Per my understanding, {{pthread_t}}
is defined in {{/usr/include/bits/pthreadtypes.h}} as
{code}
typedef unsigned long int pthread_t;
{code}
And this patch is compiled and run successfully on my Linux server.


> Implementation of true secure random with high performance using hardware random number
generator.
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-10734
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10734
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: security
>    Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>             Fix For: fs-encryption (HADOOP-10150 and HDFS-6134)
>
>         Attachments: HADOOP-10734.patch
>
>
> This JIRA is to implement Secure random using JNI to OpenSSL, and implementation should
be thread-safe.
> Utilize RdRand to return random numbers from hardware random number generator. It's TRNG(True
Random Number generators) having much higher performance than {{java.security.SecureRandom}}.

> https://wiki.openssl.org/index.php/Random_Numbers
> http://en.wikipedia.org/wiki/RdRand
> https://software.intel.com/en-us/articles/performance-impact-of-intel-secure-key-on-openssl



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message