hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-11152) Better random number generator
Date Mon, 29 Sep 2014 18:47:34 GMT
Luke Lu created HADOOP-11152:

             Summary: Better random number generator
                 Key: HADOOP-11152
                 URL: https://issues.apache.org/jira/browse/HADOOP-11152
             Project: Hadoop Common
          Issue Type: Improvement
            Reporter: Luke Lu

HDFS-7122 showed that naive ThreadLocal usage of simple LCG based j.u.Random creates unacceptable
distribution of random numbers for block placement. Similarly, ThreadLocalRandom in java 7
(same static thread local with synchronized methods overridden) has the same problem. 

"Better" is defined as better quality and faster than j.u.Random (which is already much faster
(20x) than SecureRandom).

People (e.g. Numerical Recipes) have shown that by combining LCG and XORShift we can have
a better fast RNG. It'd be worthwhile to investigate a thread local version of these "better"

This message was sent by Atlassian JIRA

View raw message