tatyana doubtsova wrote:
> I'm not sure the iterative runs should be incorporated in CC for 2 reasons:
> 1. Those runs are time consuming.
So? They are important!
Oh, note that I said "possible"  I don't mean that everyone running the
CC framework should have to run them  I mean I can setup a second
machine and just run that target, and get all the goodness of the CC
framework for free.
> 2. ant is not acceptable to organize loops.
There must be a way to fix this.
geir
>
> Thanks,
> Tanya
>
> On 11/24/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>> This is incredibly useful. Thanks. We need to incorporate this as a
>> possible target in CC.
>>
>> geir
>>
>> tatyana doubtsova wrote:
>> > I created lists of intermittent unit tests failures on Win XP in
>> perTest
>> > mode:
>> > http://issues.apache.org/jira/browse/HARMONY2282  swing module
>> > http://issues.apache.org/jira/browse/HARMONY2284  security module
>> >
>> > Could anybody take a look at it?
>> >
>> > I did not observe intermittent failures in other modules on Win XP so
>> far.
>> > I'm going to continue iterative classlib tests runs.
>> >
>> > Thanks,
>> > Tanya
>> >
>> > On 11/22/06, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
>> >
>> >> Tatiana,
>> >> That's great! Feel free to file JIRA issues about new problems.
>> >>
>> >> This is quite interesting that you haven't discovered
>> >> org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest
>> >> failure using effectively the same revision as I. This makes me think
>> >> that my problems can be local to my computer.
>> >>
>> >> 
>> >> Thank you,
>> >> Alexei
>> >>
>> >> On 11/21/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
>> >> > Hello,
>> >> >
>> >> > I've run classlib unit tests on Win XP in Other VM mode 50 times for
>> >> each
>> >> > module in order to reveal sporadic failures.
>> >> >
>> >> >
>> >> >
>> >> > The script is
>> >> >
>> >> >
>> >> >
>> >> > #!sh
>> >> >
>> >> > cat modulesList  while read moduleName; do
>> >> >
>> >> > A=0
>> >> >
>> >> > while [ "$A" lt "50" ]; do
>> >> >
>> >> > echo ITERATION $A
>> >> >
>> >> > ant 
>> >> >
>> >>
>> Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\
>>
>> >>
>> >> >
>> >> > Dbuild.module=$moduleName 
>> >> > Dhy.test.forkmode=perTest \
>> >> >
>> >> > test 2>&1  tee log_${moduleName}_${A}.txt
>> >> >
>> >> > cp rf build/test_report
>> >> build/test_report.${moduleName}_${A}
>> >> >
>> >> > A=`expr $A + 1`
>> >> >
>> >> > done
>> >> >
>> >> > done
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > $ cat moduleList
>> >> >
>> >> >
>> >> >
>> >> > accessibility
>> >> >
>> >> > archive
>> >> >
>> >> > auth
>> >> >
>> >> > awt
>> >> >
>> >> > beans
>> >> >
>> >> > crypto
>> >> >
>> >> > instrument
>> >> >
>> >> > jndi
>> >> >
>> >> > langmanagement
>> >> >
>> >> > logging
>> >> >
>> >> > luni
>> >> >
>> >> > math
>> >> >
>> >> > nio
>> >> >
>> >> > nio_char
>> >> >
>> >> > prefs
>> >> >
>> >> > print
>> >> >
>> >> > regex
>> >> >
>> >> > rmi
>> >> >
>> >> > security
>> >> >
>> >> > sound
>> >> >
>> >> > sql
>> >> >
>> >> > swing
>> >> >
>> >> > text
>> >> >
>> >> > tools
>> >> >
>> >> > xnet
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > I've observed the following failures on svn=r476124
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *Module*
>> >> >
>> >> > *Test*
>> >> >
>> >> > *Number of failures*
>> >> >
>> >> > *issue*
>> >> >
>> >> > luni
>> >> >
>> >> >
>> >>
>> org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest
>>
>> >>
>> >> >
>> >> > 50 out of 50
>> >> >
>> >> > 1879 <http://issues.apache.org/jira/browse/HARMONY1879>, updated
>> >> >
>> >> > luni
>> >> >
>> >> > tests.api.java.net.InetAddressTest
>> >> >
>> >> > 50 out of 50
>> >> >
>> >> > Test was eliminated from repository after Novenber,17
>> >> >
>> >> > prefs
>> >> >
>> >> > org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
>> >> >
>> >> > 50 out of 50
>> >> >
>> >> > 1751 <http://issues.apache.org/jira/browse/HARMONY1751>
>> >> >
>> >> > swing
>> >> >
>> >> > javax.swing.text.WrappedPlainView_SimpleTest
>> >> >
>> >> > 1 time out of 50
>> >> >
>> >> > 1)
>> >> >
>> >> > swing
>> >> >
>> >> > javax.swing.SpinnerDateModelTest
>> >> >
>> >> > 35 times out of 50
>> >> >
>> >> > 2)
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > 1). Eror
>> >> >
>> >> > Timeout occurred
>> >> >
>> >> > junit.framework.AssertionFailedError: Timeout occurred
>> >> >
>> >> >
>> >> >
>> >> > 2). Failure
>> >> >
>> >> > expected:<0> but was:<7>
>> >> >
>> >> > junit.framework.AssertionFailedError: expected:<0> but was:<7>
at
>> >> > javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
>> >> > SpinnerDateModelTest.java:59) at
>> >> > java.lang.reflect.VMReflection.invokeMethod(Native
>> >> > Method) at javax.swing.BasicSwingTestCase.runBareSuper(
>> >> > BasicSwingTestCase.java) at
>> javax.swing.BasicSwingTestCase.runBareImpl(
>> >> > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
>> >> > BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)
>> >> >
>> >> >
>> >> >
>> >> > 2070 <http://issues.apache.org/jira/browse/HARMONY2070> was
not
>> >> reproduced
>> >> > during this test run
>> >> >
>> >> >
>> >> >
>> >> > I can't reproduce 1) and 2) today on svn = r477507
>> >> >
>> >> >
>> >> >
>> >> > I've excluded failed tests and restarted multiple test run on Win XP
>> in
>> >> > order to make sure there is no other sporadic failures.
>> >> >
>> >> > Also I'm going to start the multiple classlib tests run on SUSE 9
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > To extract the results I've used commands:
>> >> >
>> >> > cd classlib/trunk/buid
>> >> >
>> >> > # find errors
>> >> >
>> >> > find . name *Test.xml exec grep l "<testsuite errors=\"[19]*\"
"
>> {}
>> >> \; >
>> >> > ERRORS
>> >> >
>> >> > # find failures
>> >> >
>> >> > find . name *Test.xml exec grep l "<testsuite errors=\"*\"
>> >> > failures=\"[19]*\"" {} \; > FAILURES
>> >> >
>> >> > find . type f size 0  grep *.xml > ZERO_LENGTH
>> >> >
>> >> >
>> >> >
>> >> > Also I've compared the number of files in test_report directories:
>> >> >
>> >> > #!sh
>> >> >
>> >> > cat modulesList  while read moduleName; do
>> >> >
>> >> > echo $moduleName
>> >> >
>> >> > A=1
>> >> >
>> >> > while [ "$A" lt "51" ]; do
>> >> >
>> >> > L=`ls 1 build/test_report.${moduleName}_${A}  grep xml
>> >>  wc
>> >> > l`
>> >> >
>> >> > echo build/test_report.${moduleName}_${A} $L >>
>> >> > test_report_length
>> >> >
>> >> > A=`expr $A + 1`
>> >> >
>> >> > done
>> >> > done
>> >> >
>> >> > Thanks,
>> >> > Tanya
>> >> >
>> >> >
>> >>
>> >
>>
>
