Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of todd@cloudera.com designates
 209.85.161.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAGWW7Lk5SSA=N==mAJcKKynC9rsUjruuAxDo4-yy8T2th_MJaQ@mail.gmail.com>
References: 
 <CADcMMgEO31OZW8ZXi2F2rqtL+nqEi13aWUT_ssqH+yqFzPQ8rw@mail.gmail.com>
 <1322269462.68973.YahooMailNeo@web121706.mail.ne1.yahoo.com>
 <CALte62w0SwNe_PM6y1G5tuBeELDS6o59QoQ0zrKEnUPM0gq9rA@mail.gmail.com>
 <CALte62wv60ws31Y4mFuWjrDWjavfckygtbvQ0Q7XnGGzkSi0Jg@mail.gmail.com>
 <CADcMMgEgLPV-1OS8FK+1wftJHDCaBsu_CjwkVtzVi3naOxv=-Q@mail.gmail.com>
 <1322283977.89408.YahooMailNeo@web121712.mail.ne1.yahoo.com>
 <CAGWW7Lk5SSA=N==mAJcKKynC9rsUjruuAxDo4-yy8T2th_MJaQ@mail.gmail.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Fri, 25 Nov 2011 22:22:35 -0800
Message-ID: 
 <CADY20s5AQjP_SsNWW25_TbS1GTWbpaR5E+=KwGKvMaXS_OPJtQ@mail.gmail.com>
Subject: Re: Status of 0.92RC
To: dev@hbase.apache.org
Cc: lars hofhansl <lhofhansl@yahoo.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

In Hadoop we use this in our pom:
            <java.security.egd>file:///dev/urandom</java.security.egd>

which also works in JDK6. See HADOOP-7841


On Fri, Nov 25, 2011 at 10:12 PM, Li Pi <li@idle.li> wrote:
> Very, very nice discovery!
>
> Once I saw the move your mouse around part, I smiled.
>
> Entropy can also be gathered from other system events, such as network
> traffic, so on a production machine, you should always have enough
> entropy. So it should be fine to change the test to use urandom.
>
> On Fri, Nov 25, 2011 at 9:06 PM, lars hofhansl <lhofhansl@yahoo.com> wrot=
e:
>> Here is something that will boggle your mind:
>> After you ran TestHCM often enough it will hang until you move your mous=
e around!
>>
>> No joke... But what sounds like an impossibility is actually related to =
the use of SecureRandom.
>> SecureRandom uses /dev/random on Linux and that gathers entropy from sys=
tem events
>> such as network activity, disk activity, keyboard activity, mouse moveme=
nt, etc.
>> If not enough the entropy is available /dev/random will block until enou=
gh entropy is gathered!
>>
>> When I add this to the command line (via pom.xml) TestHCM never hangs:
>> "-Djava.security.egd=3Dfile:/dev/./urandom"
>>
>> (note that -Djava.security.egd=3Dfile:/dev/urandom does not work for som=
e reason on JDK 1.5 or newer, =A0the extra ./ is needed)
>>
>>
>> This instructs the JDK use /dev/urandom for SecureRandom, which unlike /=
dev/random will never block, but if not enough
>> entropy exists it will generate random numbers of lesser quality... But =
for tests we don't care.
>>
>>
>> That explains why tests only time out sometimes... when the system happe=
ns to run out of entropy bits when a secure random
>> is used.
>>
>> -- Lars
>>
>>
>> ________________________________
>> From: Stack <stack@duboce.net>
>> To: dev@hbase.apache.org
>> Cc: lars hofhansl <lhofhansl@yahoo.com>
>> Sent: Friday, November 25, 2011 7:14 PM
>> Subject: Re: Status of 0.92RC
>>
>> On Fri, Nov 25, 2011 at 6:16 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> I then looped TestHCM 4 times and there was no test failure.
>>>
>>
>> Its fine on mac.=A0 On ubuntu:
>>
>> ------------------------------------------------------------------------=
-------
>> Test set: org.apache.hadoop.hbase.client.TestHCM
>> ------------------------------------------------------------------------=
-------
>> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
>> 757.578 sec <<< FAILURE!
>> testClosing(org.apache.hadoop.hbase.client.TestHCM)=A0 Time elapsed:
>> 35.34 sec=A0 <<< FAILURE!
>> java.lang.AssertionError
>> =A0 =A0 =A0 =A0 at org.junit.Assert.fail(Assert.java:92)
>> =A0 =A0 =A0 =A0 at org.junit.Assert.assertTrue(Assert.java:43)
>> =A0 =A0 =A0 =A0 at org.junit.Assert.assertFalse(Assert.java:68)
>> =A0 =A0 =A0 =A0 at org.junit.Assert.assertFalse(Assert.java:79)
>> =A0 =A0 =A0 =A0 at org.apache.hadoop.hbase.client.TestHCM.testClosing(Te=
stHCM.java:221)
>> =A0 =A0 =A0 =A0 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native M=
ethod)
>>
>> ....
>>
>> Line numbers are off because I'm messing.=A0 Its saying connection 1 is
>> closed if I test it just after creating it.
>>
>> St.Ack
>>
>>> On Fri, Nov 25, 2011 at 5:39 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>
>>>> I looped TestHCM#testClosing 5 times on MacBook and didn't see test
>>>> failure.
>>>>
>>>> Stack:
>>>> Can you share the test output ?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On Fri, Nov 25, 2011 at 5:04 PM, lars hofhansl <lhofhansl@yahoo.com>wr=
ote:
>>>>
>>>>> I added testClosing as part of HBASE-4805, I'll have a look as soon a=
s I
>>>>> get a chance.
>>>>>
>>>>>
>>>>>
>>>>> ________________________________
>>>>> =A0From: Stack <stack@duboce.net>
>>>>> To: HBase Dev List <dev@hbase.apache.org>
>>>>> Sent: Friday, November 25, 2011 2:12 PM
>>>>> Subject: Status of 0.92RC
>>>>>
>>>>> I'm having a little difficulty getting all tests to pass. =A0On
>>>>> linux/ubuntu, TestHCM (testClosing strange issue) and TestReplication
>>>>> are failing for me. =A0On mac osx, it'll build without fail about 50%=
 of
>>>>> the time. =A0I'd like to make it so tests pass all the time before
>>>>> cutting the RC. =A0Thats what I'm at these times.
>>>>>
>>>>> Also, 0.92 build on jenkins has been turned off by Apache
>>>>> Infrastructure. =A0It was hanging. =A0Its done this in the past too a=
nd
>>>>> when it hangs it requires a jenkins reboot which doesn't make Apache
>>>>> Infrastructure team too happy. =A0The hang looks to me like a Jenkins
>>>>> bug because build hangs before we even checkout src. =A0Am trying to =
see
>>>>> what can be done to get it going again but thats the story at the mo.
>>>>>
>>>>> St.Ack
>>>>>
>>>>
>>>>
>>>
>>
>


--=20
Todd Lipcon
Software Engineer, Cloudera