harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tatyana doubtsova" <tatyanadoubts...@gmail.com>
Subject [drlvm][classlib unit tests] iterative runs
Date Tue, 21 Nov 2006 14:06:37 GMT
Hello,

I've run classlib unit tests on Win XP in Other VM mode 50 times for each
module in order to reveal sporadic failures.



The script is



#!sh

cat modulesList | while read moduleName; do

            A=0

            while [ "$A" -lt "50" ]; do

            echo ITERATION $A

            ant -
Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\

                        -Dbuild.module=$moduleName -
Dhy.test.forkmode=perTest \

                        test 2>&1 | tee log_${moduleName}_${A}.txt

            cp -rf build/test_report build/test_report.${moduleName}_${A}

            A=`expr $A + 1`

            done

done





$ cat moduleList



accessibility

archive

auth

awt

beans

crypto

instrument

jndi

lang-management

logging

luni

math

nio

nio_char

prefs

print

regex

rmi

security

sound

sql

swing

text

tools

x-net





I've observed the following failures on svn=r476124





  *Module*

*Test*

*Number of failures*

*issue*

luni

org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest

50 out of 50

1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated

luni

tests.api.java.net.InetAddressTest

50 out of 50

Test was eliminated from repository after Novenber,17

prefs

org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest

50 out of 50

1751 <http://issues.apache.org/jira/browse/HARMONY-1751>

swing

javax.swing.text.WrappedPlainView_SimpleTest

1 time out of 50

1)

swing

javax.swing.SpinnerDateModelTest

35 times out of 50

2)




1). Eror

Timeout occurred

junit.framework.AssertionFailedError: Timeout occurred



2). Failure

expected:<0> but was:<7>

junit.framework.AssertionFailedError: expected:<0> but was:<7> at
javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
SpinnerDateModelTest.java:59) at
java.lang.reflect.VMReflection.invokeMethod(Native
Method) at javax.swing.BasicSwingTestCase.runBareSuper(
BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl(
BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)



2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not reproduced
during this test run



I can't reproduce 1) and 2) today on svn = r477507



I've excluded failed tests and restarted multiple test run on Win XP in
order to make sure there is no other sporadic failures.

Also I'm going to start the multiple classlib tests run on SUSE 9





To extract the results I've used commands:

cd classlib/trunk/buid

# find errors

find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {} \; >
ERRORS

# find failures

find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
failures=\"[1-9]*\"" {} \; > FAILURES

find . -type f -size 0 | grep *.xml > ZERO_LENGTH



Also I've compared the number of files in test_report directories:

#!sh

cat modulesList | while read moduleName; do

            echo $moduleName

            A=1

            while [ "$A" -lt "51" ]; do

            L=`ls -1 build/test_report.${moduleName}_${A} | grep xml | wc
-l`

            echo build/test_report.${moduleName}_${A} $L >>
test_report_length

            A=`expr $A + 1`

            done
done

Thanks,
Tanya

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message