hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Lederman <jon2...@gmail.com>
Subject Re: Entropy Pool and HDFS FS Commands Hanging System
Date Tue, 04 Jan 2011 00:48:38 GMT
Thanks.  Will try that.  One final question, based on the jstack output I sent, is it obvious
that the system is blocked due to the behavior of /dev/random?  That is, can you enlighten
me to the output I sent that explicitly or implicitly indicates the blocking?  I am trying
to understand whether this is in fact the problem or whether there could be some other issue.
 

If I just let the FS command run (i.e., hadoop fs -ls), is there any guarantee it will eventually
return in some relatively finite period of time such as hours, or could it potentially take
days, weeks, years or eternity?

Thanks in advance.

-Jon
On Jan 3, 2011, at 4:41 PM, Ted Dunning wrote:

> try
> 
>   dd if=/dev/random bs=1 count=100 of=/dev/null
> 
> This will likely hang for a long time.
> 
> There is no way that I know of to change the behavior of /dev/random except
> by changing the file itself to point to a different minor device.  That
> would be very bad form.
> 
> One think you may be able do is to pour lots of entropy into the system via
> /dev/urandom.  I was not able to demonstrate this, though, when I just tried
> that.  It would be nice if there were a config variable to set that would
> change this behavior, but right now, a code change is required (AFAIK).
> 
> Another thing to do is replace the use of SecureRandom with a version that
> uses /dev/urandom.  That is the point of the code that I linked to.  It
> provides a plugin replacement that will not block.
> 
> On Mon, Jan 3, 2011 at 4:31 PM, Jon Lederman <jon2718@gmail.com> wrote:
> 
>> 
>> Could you give me a bit more information on how I can overcome this issue.
>> I am running Hadoop on an embedded processor and networking is turned off
>> to the embedded processor. Is there a quick way to check whether this is in
>> fact blocking on my system?  And, are there some variables or configuration
>> options I can set to avoid any potential blocking behavior?
>> 
>> 


Mime
View raw message