harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [classlib][testing] intermittent tests
Date Tue, 27 Mar 2007 13:37:03 GMT
Well, the difference is that intermittently failing tests are not
excluded by default and it is expected that they will be used for
pre-commit testing. However, CC excludes intermittent tests from its
test runs.

Regards,

2007/3/27, Tony Wu <wuyuehao@gmail.com>:
> IMO we have to deal with them no matter they are intermittent failures
> or permanent failures. all of them should be excluded. we mark a
> failure as intermittent for someone who will try to fix them in the
> future, he will not be confused on getting passed results when running
> a few times.
>
> On 3/27/07, Mikhail Loenko <mloenko@gmail.com> wrote:
> > 2007/3/23, Alexei Zakharov <alexei.zakharov@gmail.com>:
> > > Hi all,
> > >
> > > In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> > > some tests fail very rarely. Say, one failure per 50 tries. So the
> > > question to all is: should this be treated as an intermittent failure?
> > > Thoughts?
> >
> > Do you think it is a permanent failure? :)
> >
> > if it's still a failure, we should not run the tests under CCs to avoid
> > false alarms
> >
> > Thanks,
> > Mikhail
> >
> >
> > >
> > > Thanks,
> > >
> > > 2007/3/22, tatyana doubtsova (JIRA) <jira@apache.org>:
> > > >
> > > >     [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130
]
> > > >
> > > > tatyana doubtsova commented on HARMONY-1979:
> > > > --------------------------------------------
> > > >
> > > > Alexei, the behavior of awt module tests is unstable on linux ia32, ia
64. New runs show new failures.
> > > >
> > > > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> > > >
> > > > 1. java.awt.WindowTest.testSetLocationRelativeTo     fails 6 times out
of 50
> > > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]>
but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89)
at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > >
> > > > 2. java.awt.event.InvocationEventTest   - possible crash 1 out of 50
> > > >
> > > > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out
of 50
> > > >     org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes
2 out of 50
> > > >       java.lang.OutOfMemoryError <no stack trace available>
> > > >
> > > > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> > > >
> > > > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> > > >
> > > > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> > > >
> > > > 1. java.awt.EventDispatchThreadRTest - possible crash  4 out of 30
> > > > 2. java.awt.AccessibleAWTWindowTest  - possible crash 9 out of 30
> > > > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > > > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of
 30
> > > > 5. java.awt.BorderLayoutRTest - possible crash 1 out of  30
> > > > 6. java.awt.WindowTest.testSetLocationRelativeTo  fails 8 times out of
30
> > > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]>
but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89)
at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString
 failes 1 time out of 30
> > > > junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152)
at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > >
> > > >
> > > >
> > > > > [classlib][awt][test] Many excluded AWT tests pass
> > > > > --------------------------------------------------
> > > > >
> > > > >                 Key: HARMONY-1979
> > > > >                 URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > > >             Project: Harmony
> > > > >          Issue Type: Bug
> > > > >          Components: build - test - ci
> > > > >            Reporter: Alexey A. Ivanov
> > > > >         Assigned To: Alexei Zakharov
> > > > >         Attachments: H1979-awt-unexclude-tests.patch
> > > > >
> > > > >
> > > > > Many excluded AWT tests pass successfully without any modification.
I tried both Windows and Linux.
> > > >
> > > > --
> > > > This message is automatically generated by JIRA.
> > > > -
> > > > You can reply to this email to add a comment to the issue online.

-- 
Alexei Zakharov,
Intel ESSD

Mime
View raw message