harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: [drlvm][classlib unit tests] iterative runs
Date Wed, 22 Nov 2006 00:02:50 GMT
Tatiana,
That's great! Feel free to file JIRA issues about new problems.

This is quite interesting that you haven't discovered
org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest
failure using effectively the same revision as I.  This makes me think
that my problems can be local to my computer.

-- 
Thank you,
Alexei

On 11/21/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
> Hello,
>
> I've run classlib unit tests on Win XP in Other VM mode 50 times for each
> module in order to reveal sporadic failures.
>
>
>
> The script is
>
>
>
> #!sh
>
> cat modulesList | while read moduleName; do
>
>            A=0
>
>            while [ "$A" -lt "50" ]; do
>
>            echo ITERATION $A
>
>            ant -
> Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\
>
>                        -Dbuild.module=$moduleName -
> Dhy.test.forkmode=perTest \
>
>                        test 2>&1 | tee log_${moduleName}_${A}.txt
>
>            cp -rf build/test_report build/test_report.${moduleName}_${A}
>
>            A=`expr $A + 1`
>
>            done
>
> done
>
>
>
>
>
> $ cat moduleList
>
>
>
> accessibility
>
> archive
>
> auth
>
> awt
>
> beans
>
> crypto
>
> instrument
>
> jndi
>
> lang-management
>
> logging
>
> luni
>
> math
>
> nio
>
> nio_char
>
> prefs
>
> print
>
> regex
>
> rmi
>
> security
>
> sound
>
> sql
>
> swing
>
> text
>
> tools
>
> x-net
>
>
>
>
>
> I've observed the following failures on svn=r476124
>
>
>
>
>
>  *Module*
>
> *Test*
>
> *Number of failures*
>
> *issue*
>
> luni
>
> org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest
>
> 50 out of 50
>
> 1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated
>
> luni
>
> tests.api.java.net.InetAddressTest
>
> 50 out of 50
>
> Test was eliminated from repository after Novenber,17
>
> prefs
>
> org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
>
> 50 out of 50
>
> 1751 <http://issues.apache.org/jira/browse/HARMONY-1751>
>
> swing
>
> javax.swing.text.WrappedPlainView_SimpleTest
>
> 1 time out of 50
>
> 1)
>
> swing
>
> javax.swing.SpinnerDateModelTest
>
> 35 times out of 50
>
> 2)
>
>
>
>
> 1). Eror
>
> Timeout occurred
>
> junit.framework.AssertionFailedError: Timeout occurred
>
>
>
> 2). Failure
>
> expected:<0> but was:<7>
>
> junit.framework.AssertionFailedError: expected:<0> but was:<7> at
> javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
> SpinnerDateModelTest.java:59) at
> java.lang.reflect.VMReflection.invokeMethod(Native
> Method) at javax.swing.BasicSwingTestCase.runBareSuper(
> BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl(
> BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
> BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)
>
>
>
> 2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not reproduced
> during this test run
>
>
>
> I can't reproduce 1) and 2) today on svn = r477507
>
>
>
> I've excluded failed tests and restarted multiple test run on Win XP in
> order to make sure there is no other sporadic failures.
>
> Also I'm going to start the multiple classlib tests run on SUSE 9
>
>
>
>
>
> To extract the results I've used commands:
>
> cd classlib/trunk/buid
>
> # find errors
>
> find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {} \; >
> ERRORS
>
> # find failures
>
> find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
> failures=\"[1-9]*\"" {} \; > FAILURES
>
> find . -type f -size 0 | grep *.xml > ZERO_LENGTH
>
>
>
> Also I've compared the number of files in test_report directories:
>
> #!sh
>
> cat modulesList | while read moduleName; do
>
>            echo $moduleName
>
>            A=1
>
>            while [ "$A" -lt "51" ]; do
>
>            L=`ls -1 build/test_report.${moduleName}_${A} | grep xml | wc
> -l`
>
>            echo build/test_report.${moduleName}_${A} $L >>
> test_report_length
>
>            A=`expr $A + 1`
>
>            done
> done
>
> Thanks,
> Tanya
>
>

Mime
View raw message