harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: [drlvm][classlib unit tests] iterative runs
Date Mon, 27 Nov 2006 21:14:16 GMT
Tatiana,
Thanks for updating the issue with new failures. I believe it is very
important to have intermittent failures documented.

Next time I would suggest filing a new issue in case when a resolution
provider solves at least a part of the problem. Why?

* The resolution provider is happy to have the issue solved by him.
* All other readers including a resolution provider and committer for
another part of the issue have less items to read and understand.

My best practice is to reopen issues only if a proposed resolution is
not a step in the right direction.

Thank you again for adding information about new intermittent failures
- this is much more valuable than my previous comment.
-- 
Thank you,
Alexei


On 11/27/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
> There are intermittent failures in nio module after applying fix to
> http://issues.apache.org/jira/browse/HARMONY-2285. I udated the issue.
>
> Thanks,
> Tanya
>
>
> On 11/24/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
> >
> > I also created issues regarding intermittent failures on SUSE9:
> > http://issues.apache.org/jira/browse/HARMONY-2315
> > http://issues.apache.org/jira/browse/HARMONY-2307
> >
> > Thanks,
> > Tanya
> >
> > On 11/23/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
> > >
> > > I created lists of intermittent unit tests failures on Win XP in perTest
> > > mode:
> > > http://issues.apache.org/jira/browse/HARMONY-2282 - swing module
> > > http://issues.apache.org/jira/browse/HARMONY-2284 - security module
> > >
> > > Could anybody take a look at it?
> > >
> > > I did not observe intermittent failures in other modules on Win XP so
> > > far.
> > > I'm going to continue iterative classlib tests runs.
> > >
> > > Thanks,
> > > Tanya
> > >
> > > On 11/22/06, Alexei Fedotov <alexei.fedotov@gmail.com > wrote:
> > >
> > > > Tatiana,
> > > > That's great! Feel free to file JIRA issues about new problems.
> > > >
> > > > This is quite interesting that you haven't discovered
> > > > org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest
> > > > failure using effectively the same revision as I.  This makes me think
> > > > that my problems can be local to my computer.
> > > >
> > > > --
> > > > Thank you,
> > > > Alexei
> > > >
> > > > On 11/21/06, tatyana doubtsova <tatyanadoubtsova@gmail.com> wrote:
> > > > > Hello,
> > > > >
> > > > > I've run classlib unit tests on Win XP in Other VM mode 50 times
for
> > > > each
> > > > > module in order to reveal sporadic failures.
> > > > >
> > > > >
> > > > >
> > > > > The script is
> > > > >
> > > > >
> > > > >
> > > > > #!sh
> > > > >
> > > > > cat modulesList | while read moduleName; do
> > > > >
> > > > >            A=0
> > > > >
> > > > >            while [ "$A" -lt "50" ]; do
> > > > >
> > > > >            echo ITERATION $A
> > > > >
> > > > >            ant -
> > > > >
> > > > Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\
> > > > >
> > > > >                        -Dbuild.module=$moduleName -
> > > > > Dhy.test.forkmode=perTest \
> > > > >
> > > > >                        test 2>&1 | tee log_${moduleName}_${A}.txt
> > > > >
> > > > >            cp -rf build/test_report
> > > > build/test_report.${moduleName}_${A}
> > > > >
> > > > >            A=`expr $A + 1`
> > > > >
> > > > >            done
> > > > >
> > > > > done
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > $ cat moduleList
> > > > >
> > > > >
> > > > >
> > > > > accessibility
> > > > >
> > > > > archive
> > > > >
> > > > > auth
> > > > >
> > > > > awt
> > > > >
> > > > > beans
> > > > >
> > > > > crypto
> > > > >
> > > > > instrument
> > > > >
> > > > > jndi
> > > > >
> > > > > lang-management
> > > > >
> > > > > logging
> > > > >
> > > > > luni
> > > > >
> > > > > math
> > > > >
> > > > > nio
> > > > >
> > > > > nio_char
> > > > >
> > > > > prefs
> > > > >
> > > > > print
> > > > >
> > > > > regex
> > > > >
> > > > > rmi
> > > > >
> > > > > security
> > > > >
> > > > > sound
> > > > >
> > > > > sql
> > > > >
> > > > > swing
> > > > >
> > > > > text
> > > > >
> > > > > tools
> > > > >
> > > > > x-net
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I've observed the following failures on svn=r476124
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  *Module*
> > > > >
> > > > > *Test*
> > > > >
> > > > > *Number of failures*
> > > > >
> > > > > *issue*
> > > > >
> > > > > luni
> > > > >
> > > > >
> > > > org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest
> > > > >
> > > > > 50 out of 50
> > > > >
> > > > > 1879 < http://issues.apache.org/jira/browse/HARMONY-1879>,
updated
> > > > >
> > > > > luni
> > > > >
> > > > > tests.api.java.net.InetAddressTest
> > > > >
> > > > > 50 out of 50
> > > > >
> > > > > Test was eliminated from repository after Novenber,17
> > > > >
> > > > > prefs
> > > > >
> > > > > org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
> > > > >
> > > > > 50 out of 50
> > > > >
> > > > > 1751 <http://issues.apache.org/jira/browse/HARMONY-1751 >
> > > > >
> > > > > swing
> > > > >
> > > > > javax.swing.text.WrappedPlainView_SimpleTest
> > > > >
> > > > > 1 time out of 50
> > > > >
> > > > > 1)
> > > > >
> > > > > swing
> > > > >
> > > > > javax.swing.SpinnerDateModelTest
> > > > >
> > > > > 35 times out of 50
> > > > >
> > > > > 2)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > 1). Eror
> > > > >
> > > > > Timeout occurred
> > > > >
> > > > > junit.framework.AssertionFailedError: Timeout occurred
> > > > >
> > > > >
> > > > >
> > > > > 2). Failure
> > > > >
> > > > > expected:<0> but was:<7>
> > > > >
> > > > > junit.framework.AssertionFailedError : expected:<0> but was:<7>
at
> > > > > javax.swing.SpinnerDateModelTest.testSpinnerDateModel (
> > > > > SpinnerDateModelTest.java:59) at
> > > > > java.lang.reflect.VMReflection.invokeMethod(Native
> > > > > Method) at javax.swing.BasicSwingTestCase.runBareSuper(
> > > > > BasicSwingTestCase.java) at
> > > > javax.swing.BasicSwingTestCase.runBareImpl (
> > > > > BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
> > > > > BasicSwingTestCase.java ) at java.lang.Thread.run(Unknown Source)
> > > > >
> > > > >
> > > > >
> > > > > 2070 < http://issues.apache.org/jira/browse/HARMONY-2070> was
not
> > > > reproduced
> > > > > during this test run
> > > > >
> > > > >
> > > > >
> > > > > I can't reproduce 1) and 2) today on svn = r477507
> > > > >
> > > > >
> > > > >
> > > > > I've excluded failed tests and restarted multiple test run on Win
XP
> > > > in
> > > > > order to make sure there is no other sporadic failures.
> > > > >
> > > > > Also I'm going to start the multiple classlib tests run on SUSE 9
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > To extract the results I've used commands:
> > > > >
> > > > > cd classlib/trunk/buid
> > > > >
> > > > > # find errors
> > > > >
> > > > > find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\"
"
> > > > {} \; >
> > > > > ERRORS
> > > > >
> > > > > # find failures
> > > > >
> > > > > find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
> > > > > failures=\"[1-9]*\"" {} \; > FAILURES
> > > > >
> > > > > find . -type f -size 0 | grep *.xml > ZERO_LENGTH
> > > > >
> > > > >
> > > > >
> > > > > Also I've compared the number of files in test_report directories:
> > > > >
> > > > > #!sh
> > > > >
> > > > > cat modulesList | while read moduleName; do
> > > > >
> > > > >            echo $moduleName
> > > > >
> > > > >            A=1
> > > > >
> > > > >            while [ "$A" -lt "51" ]; do
> > > > >
> > > > >            L=`ls -1 build/test_report.${moduleName}_${A} | grep xml
> > > > | wc
> > > > > -l`
> > > > >
> > > > >            echo build/test_report.${moduleName}_${A} $L >>
> > > > > test_report_length
> > > > >
> > > > >            A=`expr $A + 1`
> > > > >
> > > > >            done
> > > > > done
> > > > >
> > > > > Thanks,
> > > > > Tanya
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
>

Mime
View raw message