hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Entropy Pool and HDFS FS Commands Hanging System
Date Tue, 04 Jan 2011 04:47:33 GMT
Another possibility to fix it is to install rng-tools which will allow
you to increase the amount of entropy in your system.
  Take care,
Konstantin (Cos) Boudnik

On Mon, Jan 3, 2011 at 16:48, Jon Lederman <jon2718@gmail.com> wrote:
> Thanks.  Will try that.  One final question, based on the jstack output I sent, is
it obvious that the system is blocked due to the behavior of /dev/random?  That is, can you
enlighten me to the output I sent that explicitly or implicitly indicates the blocking?  I
am trying to understand whether this is in fact the problem or whether there could be some
other issue.
> If I just let the FS command run (i.e., hadoop fs -ls), is there any guarantee it will
eventually return in some relatively finite period of time such as hours, or could it potentially
take days, weeks, years or eternity?
> Thanks in advance.
> -Jon
> On Jan 3, 2011, at 4:41 PM, Ted Dunning wrote:
>> try
>>   dd if=/dev/random bs=1 count=100 of=/dev/null
>> This will likely hang for a long time.
>> There is no way that I know of to change the behavior of /dev/random except
>> by changing the file itself to point to a different minor device.  That
>> would be very bad form.
>> One think you may be able do is to pour lots of entropy into the system via
>> /dev/urandom.  I was not able to demonstrate this, though, when I just tried
>> that.  It would be nice if there were a config variable to set that would
>> change this behavior, but right now, a code change is required (AFAIK).
>> Another thing to do is replace the use of SecureRandom with a version that
>> uses /dev/urandom.  That is the point of the code that I linked to.  It
>> provides a plugin replacement that will not block.
>> On Mon, Jan 3, 2011 at 4:31 PM, Jon Lederman <jon2718@gmail.com> wrote:
>>> Could you give me a bit more information on how I can overcome this issue.
>>> I am running Hadoop on an embedded processor and networking is turned off
>>> to the embedded processor. Is there a quick way to check whether this is in
>>> fact blocking on my system?  And, are there some variables or configuration
>>> options I can set to avoid any potential blocking behavior?

View raw message