hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Lederman <jon2...@gmail.com>
Subject Re: Entropy Pool and HDFS FS Commands Hanging System
Date Tue, 04 Jan 2011 00:31:50 GMT
Hi Ted,

Could you give me a bit more information on how I can overcome this issue.  I am running Hadoop
on an embedded processor and networking is turned off to the embedded processor. Is there
a quick way to check whether this is in fact blocking on my system?  And, are there some variables
or configuration options I can set to avoid any potential blocking behavior?

Thanks.

-Jon
On Jan 3, 2011, at 3:48 PM, Ted Dunning wrote:

> Yes.  It is stuck as suggested.  See the bolded lines.
> 
> You can help avoid this by dumping additional entropy into the machine via
> network traffic.  According to the man page for /dev/random you can cheat by
> writing goo into /dev/urandom, but I have been unable to verify that by
> experiment.
> 
> Is it really necessary to use /dev/random here?  Again from the man page,
> there is a strong feeling in the community that only very long lived, high
> value keys really need to read from /dev/random.  Session keys from
> /dev/urandom are fine.
> 
> I wrote an adaptation of the secure seed generator that doesn't block for
> Mahout.  It is trivial, but might be useful to copy:
> http://svn.apache.org/repos/asf/mahout/trunk/math/src/main/java/org/apache/mahout/common/DevURandomSeedGenerator.java
> 
> 
> 
> On Mon, Jan 3, 2011 at 3:13 PM, Jon Lederman <jon2718@gmail.com> wrote:
> 
>> I have attached the jstack <pid of namenode> output.  Does it appear to be
>> stuck in SecureRandom as you noted as a possibility?  I am not sure whether
>> this is indicated in the following output:
>> 
>> ...
>> 
> "main" prio=10 tid=0x000583c8 nid=0xf3f runnable [0xb729d000]
>>  java.lang.Thread.State: RUNNABLE
>> *        at java.io.FileInputStream.readBytes(Native Method)
>> *        at java.io.FileInputStream.read(FileInputStream.java:236)
>>       at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
>>       at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
>>       - locked <0x70e59ae8> (a java.io.BufferedInputStream)
>>       at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>>       at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
>>       at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
>>       - locked <0x70e59970> (a java.io.BufferedInputStream)
>>       at
>> sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte(SeedGenerator.java:469)
>>       at
>> sun.security.provider.SeedGenerator.getSeedBytes(SeedGenerator.java:140)
>>       at
>> sun.security.provider.SeedGenerator.generateSeed(SeedGenerator.java:135)
>> *        at
>> sun.security.provider.SecureRandom.engineGenerateSeed(SecureRandom.java:131)
>> *        at
>> sun.security.provider.SecureRandom.engineNextBytes(SecureRandom.java:188)
>> 
>> 


Mime
View raw message