harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [drlvm][classlib unit tests] iterative runs
Date Fri, 24 Nov 2006 16:33:20 GMT
This is incredibly useful.  Thanks.  We need to incorporate this as a 
possible target in CC.

geir

tatyana doubtsova wrote:
> I created lists of intermittent unit tests failures on Win XP in perTest
> mode:
> http://issues.apache.org/jira/browse/HARMONY-2282 - swing module
> http://issues.apache.org/jira/browse/HARMONY-2284 - security module
> 
> Could anybody take a look at it?
> 
> I did not observe intermittent failures in other modules on Win XP so far.
> I'm going to continue iterative classlib tests runs.
> 
> Thanks,
> Tanya
> 
> On 11/22/06, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> 
>> Tatiana,
>> That's great! Feel free to file JIRA issues about new problems.
>>
>> This is quite interesting that you haven't discovered
>> org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest
>> failure using effectively the same revision as I.  This makes me think
>> that my problems can be local to my computer.
>>
>> -- 
>> Thank you,
>> Alexei
>>
>> On 11/21/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
>> > Hello,
>> >
>> > I've run classlib unit tests on Win XP in Other VM mode 50 times for
>> each
>> > module in order to reveal sporadic failures.
>> >
>> >
>> >
>> > The script is
>> >
>> >
>> >
>> > #!sh
>> >
>> > cat modulesList | while read moduleName; do
>> >
>> >            A=0
>> >
>> >            while [ "$A" -lt "50" ]; do
>> >
>> >            echo ITERATION $A
>> >
>> >            ant -
>> >
>> Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\

>>
>> >
>> >                        -Dbuild.module=$moduleName -
>> > Dhy.test.forkmode=perTest \
>> >
>> >                        test 2>&1 | tee log_${moduleName}_${A}.txt
>> >
>> >            cp -rf build/test_report 
>> build/test_report.${moduleName}_${A}
>> >
>> >            A=`expr $A + 1`
>> >
>> >            done
>> >
>> > done
>> >
>> >
>> >
>> >
>> >
>> > $ cat moduleList
>> >
>> >
>> >
>> > accessibility
>> >
>> > archive
>> >
>> > auth
>> >
>> > awt
>> >
>> > beans
>> >
>> > crypto
>> >
>> > instrument
>> >
>> > jndi
>> >
>> > lang-management
>> >
>> > logging
>> >
>> > luni
>> >
>> > math
>> >
>> > nio
>> >
>> > nio_char
>> >
>> > prefs
>> >
>> > print
>> >
>> > regex
>> >
>> > rmi
>> >
>> > security
>> >
>> > sound
>> >
>> > sql
>> >
>> > swing
>> >
>> > text
>> >
>> > tools
>> >
>> > x-net
>> >
>> >
>> >
>> >
>> >
>> > I've observed the following failures on svn=r476124
>> >
>> >
>> >
>> >
>> >
>> >  *Module*
>> >
>> > *Test*
>> >
>> > *Number of failures*
>> >
>> > *issue*
>> >
>> > luni
>> >
>> >
>> org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest 
>>
>> >
>> > 50 out of 50
>> >
>> > 1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated
>> >
>> > luni
>> >
>> > tests.api.java.net.InetAddressTest
>> >
>> > 50 out of 50
>> >
>> > Test was eliminated from repository after Novenber,17
>> >
>> > prefs
>> >
>> > org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
>> >
>> > 50 out of 50
>> >
>> > 1751 <http://issues.apache.org/jira/browse/HARMONY-1751>
>> >
>> > swing
>> >
>> > javax.swing.text.WrappedPlainView_SimpleTest
>> >
>> > 1 time out of 50
>> >
>> > 1)
>> >
>> > swing
>> >
>> > javax.swing.SpinnerDateModelTest
>> >
>> > 35 times out of 50
>> >
>> > 2)
>> >
>> >
>> >
>> >
>> > 1). Eror
>> >
>> > Timeout occurred
>> >
>> > junit.framework.AssertionFailedError: Timeout occurred
>> >
>> >
>> >
>> > 2). Failure
>> >
>> > expected:<0> but was:<7>
>> >
>> > junit.framework.AssertionFailedError: expected:<0> but was:<7> at
>> > javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
>> > SpinnerDateModelTest.java:59) at
>> > java.lang.reflect.VMReflection.invokeMethod(Native
>> > Method) at javax.swing.BasicSwingTestCase.runBareSuper(
>> > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl(
>> > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
>> > BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)
>> >
>> >
>> >
>> > 2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not
>> reproduced
>> > during this test run
>> >
>> >
>> >
>> > I can't reproduce 1) and 2) today on svn = r477507
>> >
>> >
>> >
>> > I've excluded failed tests and restarted multiple test run on Win XP in
>> > order to make sure there is no other sporadic failures.
>> >
>> > Also I'm going to start the multiple classlib tests run on SUSE 9
>> >
>> >
>> >
>> >
>> >
>> > To extract the results I've used commands:
>> >
>> > cd classlib/trunk/buid
>> >
>> > # find errors
>> >
>> > find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {}
>> \; >
>> > ERRORS
>> >
>> > # find failures
>> >
>> > find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
>> > failures=\"[1-9]*\"" {} \; > FAILURES
>> >
>> > find . -type f -size 0 | grep *.xml > ZERO_LENGTH
>> >
>> >
>> >
>> > Also I've compared the number of files in test_report directories:
>> >
>> > #!sh
>> >
>> > cat modulesList | while read moduleName; do
>> >
>> >            echo $moduleName
>> >
>> >            A=1
>> >
>> >            while [ "$A" -lt "51" ]; do
>> >
>> >            L=`ls -1 build/test_report.${moduleName}_${A} | grep xml 
>> | wc
>> > -l`
>> >
>> >            echo build/test_report.${moduleName}_${A} $L >>
>> > test_report_length
>> >
>> >            A=`expr $A + 1`
>> >
>> >            done
>> > done
>> >
>> > Thanks,
>> > Tanya
>> >
>> >
>>
> 

Mime
View raw message