hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Lederman <jon2...@gmail.com>
Subject Re: Entropy Pool and HDFS FS Commands Hanging System
Date Tue, 04 Jan 2011 00:48:38 GMT
Thanks.  Will try that.  One final question, based on the jstack output I sent, is it obvious
that the system is blocked due to the behavior of /dev/random?  That is, can you enlighten
me to the output I sent that explicitly or implicitly indicates the blocking?  I am trying
to understand whether this is in fact the problem or whether there could be some other issue.

If I just let the FS command run (i.e., hadoop fs -ls), is there any guarantee it will eventually
return in some relatively finite period of time such as hours, or could it potentially take
days, weeks, years or eternity?

Thanks in advance.

On Jan 3, 2011, at 4:41 PM, Ted Dunning wrote:

> try
>   dd if=/dev/random bs=1 count=100 of=/dev/null
> This will likely hang for a long time.
> There is no way that I know of to change the behavior of /dev/random except
> by changing the file itself to point to a different minor device.  That
> would be very bad form.
> One think you may be able do is to pour lots of entropy into the system via
> /dev/urandom.  I was not able to demonstrate this, though, when I just tried
> that.  It would be nice if there were a config variable to set that would
> change this behavior, but right now, a code change is required (AFAIK).
> Another thing to do is replace the use of SecureRandom with a version that
> uses /dev/urandom.  That is the point of the code that I linked to.  It
> provides a plugin replacement that will not block.
> On Mon, Jan 3, 2011 at 4:31 PM, Jon Lederman <jon2718@gmail.com> wrote:
>> Could you give me a bit more information on how I can overcome this issue.
>> I am running Hadoop on an embedded processor and networking is turned off
>> to the embedded processor. Is there a quick way to check whether this is in
>> fact blocking on my system?  And, are there some variables or configuration
>> options I can set to avoid any potential blocking behavior?

View raw message