harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tatyana doubtsova" <tatyanadoubts...@gmail.com>
Subject Re: [drlvm][classlib unit tests] iterative runs
Date Mon, 27 Nov 2006 14:33:37 GMT
I'm not sure the iterative runs should be incorporated in CC for 2 reasons:
1. Those runs are time consuming.
2. ant is not acceptable to organize loops.

Thanks,
Tanya

On 11/24/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
> This is incredibly useful.  Thanks.  We need to incorporate this as a
> possible target in CC.
>
> geir
>
> tatyana doubtsova wrote:
> > I created lists of intermittent unit tests failures on Win XP in perTest
> > mode:
> > http://issues.apache.org/jira/browse/HARMONY-2282 - swing module
> > http://issues.apache.org/jira/browse/HARMONY-2284 - security module
> >
> > Could anybody take a look at it?
> >
> > I did not observe intermittent failures in other modules on Win XP so
> far.
> > I'm going to continue iterative classlib tests runs.
> >
> > Thanks,
> > Tanya
> >
> > On 11/22/06, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> >
> >> Tatiana,
> >> That's great! Feel free to file JIRA issues about new problems.
> >>
> >> This is quite interesting that you haven't discovered
> >> org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest
> >> failure using effectively the same revision as I.  This makes me think
> >> that my problems can be local to my computer.
> >>
> >> --
> >> Thank you,
> >> Alexei
> >>
> >> On 11/21/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
> >> > Hello,
> >> >
> >> > I've run classlib unit tests on Win XP in Other VM mode 50 times for
> >> each
> >> > module in order to reveal sporadic failures.
> >> >
> >> >
> >> >
> >> > The script is
> >> >
> >> >
> >> >
> >> > #!sh
> >> >
> >> > cat modulesList | while read moduleName; do
> >> >
> >> >            A=0
> >> >
> >> >            while [ "$A" -lt "50" ]; do
> >> >
> >> >            echo ITERATION $A
> >> >
> >> >            ant -
> >> >
> >>
> Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\
> >>
> >> >
> >> >                        -Dbuild.module=$moduleName -
> >> > Dhy.test.forkmode=perTest \
> >> >
> >> >                        test 2>&1 | tee log_${moduleName}_${A}.txt
> >> >
> >> >            cp -rf build/test_report
> >> build/test_report.${moduleName}_${A}
> >> >
> >> >            A=`expr $A + 1`
> >> >
> >> >            done
> >> >
> >> > done
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > $ cat moduleList
> >> >
> >> >
> >> >
> >> > accessibility
> >> >
> >> > archive
> >> >
> >> > auth
> >> >
> >> > awt
> >> >
> >> > beans
> >> >
> >> > crypto
> >> >
> >> > instrument
> >> >
> >> > jndi
> >> >
> >> > lang-management
> >> >
> >> > logging
> >> >
> >> > luni
> >> >
> >> > math
> >> >
> >> > nio
> >> >
> >> > nio_char
> >> >
> >> > prefs
> >> >
> >> > print
> >> >
> >> > regex
> >> >
> >> > rmi
> >> >
> >> > security
> >> >
> >> > sound
> >> >
> >> > sql
> >> >
> >> > swing
> >> >
> >> > text
> >> >
> >> > tools
> >> >
> >> > x-net
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > I've observed the following failures on svn=r476124
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >  *Module*
> >> >
> >> > *Test*
> >> >
> >> > *Number of failures*
> >> >
> >> > *issue*
> >> >
> >> > luni
> >> >
> >> >
> >>
> org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest
> >>
> >> >
> >> > 50 out of 50
> >> >
> >> > 1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated
> >> >
> >> > luni
> >> >
> >> > tests.api.java.net.InetAddressTest
> >> >
> >> > 50 out of 50
> >> >
> >> > Test was eliminated from repository after Novenber,17
> >> >
> >> > prefs
> >> >
> >> > org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
> >> >
> >> > 50 out of 50
> >> >
> >> > 1751 <http://issues.apache.org/jira/browse/HARMONY-1751>
> >> >
> >> > swing
> >> >
> >> > javax.swing.text.WrappedPlainView_SimpleTest
> >> >
> >> > 1 time out of 50
> >> >
> >> > 1)
> >> >
> >> > swing
> >> >
> >> > javax.swing.SpinnerDateModelTest
> >> >
> >> > 35 times out of 50
> >> >
> >> > 2)
> >> >
> >> >
> >> >
> >> >
> >> > 1). Eror
> >> >
> >> > Timeout occurred
> >> >
> >> > junit.framework.AssertionFailedError: Timeout occurred
> >> >
> >> >
> >> >
> >> > 2). Failure
> >> >
> >> > expected:<0> but was:<7>
> >> >
> >> > junit.framework.AssertionFailedError: expected:<0> but was:<7>
at
> >> > javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
> >> > SpinnerDateModelTest.java:59) at
> >> > java.lang.reflect.VMReflection.invokeMethod(Native
> >> > Method) at javax.swing.BasicSwingTestCase.runBareSuper(
> >> > BasicSwingTestCase.java) at
> javax.swing.BasicSwingTestCase.runBareImpl(
> >> > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
> >> > BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)
> >> >
> >> >
> >> >
> >> > 2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not
> >> reproduced
> >> > during this test run
> >> >
> >> >
> >> >
> >> > I can't reproduce 1) and 2) today on svn = r477507
> >> >
> >> >
> >> >
> >> > I've excluded failed tests and restarted multiple test run on Win XP
> in
> >> > order to make sure there is no other sporadic failures.
> >> >
> >> > Also I'm going to start the multiple classlib tests run on SUSE 9
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > To extract the results I've used commands:
> >> >
> >> > cd classlib/trunk/buid
> >> >
> >> > # find errors
> >> >
> >> > find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" "
> {}
> >> \; >
> >> > ERRORS
> >> >
> >> > # find failures
> >> >
> >> > find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
> >> > failures=\"[1-9]*\"" {} \; > FAILURES
> >> >
> >> > find . -type f -size 0 | grep *.xml > ZERO_LENGTH
> >> >
> >> >
> >> >
> >> > Also I've compared the number of files in test_report directories:
> >> >
> >> > #!sh
> >> >
> >> > cat modulesList | while read moduleName; do
> >> >
> >> >            echo $moduleName
> >> >
> >> >            A=1
> >> >
> >> >            while [ "$A" -lt "51" ]; do
> >> >
> >> >            L=`ls -1 build/test_report.${moduleName}_${A} | grep xml
> >> | wc
> >> > -l`
> >> >
> >> >            echo build/test_report.${moduleName}_${A} $L >>
> >> > test_report_length
> >> >
> >> >            A=`expr $A + 1`
> >> >
> >> >            done
> >> > done
> >> >
> >> > Thanks,
> >> > Tanya
> >> >
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message