harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [general] aiming no regression
Date Fri, 22 Dec 2006 11:17:00 GMT
The issue 2845 was created to easily exclude tests by module base.
 Thanks, Vladimir


On 12/22/06, Vladimir Ivanov <ivavladimir@gmail.com> wrote:
>
> On 12/21/06, Mikhail Loenko <mloenko@gmail.com > wrote:
> >
> > 2006/12/20, Geir Magnusson Jr. < geir@pobox.com>:
> > >
> > >
> > > Vladimir Ivanov wrote:
> > > > On 12/20/06, Geir Magnusson Jr. < geir@pobox.com> wrote:
> > > >>
> > > >>
> > > >>
> > > >> Mikhail Loenko wrote:
> > > >> > Instead we may remove all swing tests from CC when run on J9
and
> > try to
> > > >> > fix the
> > > >> > problem
> > > >>
> > > >> +1
> > > >>
> > > >> geir
> > > >
> > > >
> > > >
> > > > To remove tests from CC only we need a special exclude lists for CC.
> > Is it
> > > > OK to store it together with modules exclude lists to update it with
> > > > classlib ws?
> > >
> > > I guess my question is "why is CC special?" IOW, shouldn't we exclude
> > > those tests for J9 in general?
> >
> > I'm +1 for special exclude lists for CC:
> >
> > if we exclude something in regular x-list, we lose that tests for some
> > time,
> > but when we exclude intermittent failures in CC we still can use the
> > tests
> > to check for stable failures.
> >
> > Example: seems like currently each swing test can intermittently fail
> > for some reason. We can exclude all swing tests from CC to have
> > reliable CC reports. But
> > if we exclude them from pre-commit testing we won't be able to test
> > any commit to swing.
>
>
>
> I'm +0 for special exclude lists for CC but lets make decision!
>
> While no decision I will update common exclude list locally.
>
>
>
> By the way, I prefer to extend classlib build to exclude tests for any
> module from test run. It is solve the problem with intermittent failures
> under CC and will not create new with support of CC exclude lists.
>
>
>
>  thanks, Vladimir
>
>
> Thanks,
> > Mikhail
> >
> > >
> > > >
> > > >
> > > >
> > > > Also option in the build to exclude some modules will be very
> > useful.
> > > > Something like '-Dnot.build.module=swing'.
> > > >
> > > >
> > > >
> > > > Thanks, Vladimir
> > > >
> > > >
> > > >
> > > >> >
> > > >> > Thanks,
> > > >> > Mikhail
> > > >> >
> > > >> >
> > > >> > 2006/12/20, Vladimir Ivanov < ivavladimir@gmail.com>:
> > > >> >> Actually, I was able to see these failures on swing tests
only.
> > But
> > > >> >> even for
> > > >> >> swing these failures reproduced intermittently and only when
all
> > swing
> > > >> >> tests
> > > >> >> run in the one VM.
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >>  Thanks, Vladimir
> > > >> >>
> > > >> >>
> > > >> >> On 12/19/06, Mikhail Loenko < mloenko@gmail.com> wrote:
> > > >> >> >
> > > >> >> > have you seen this stack when other tests run? maybe
gui
> > > >> >> > breaks something causing the failure? Are you able to
> > reproduce the
> > > >> >> > problem?
> > > >> >> >
> > > >> >> > Thanks,
> > > >> >> > Mikhail
> > > >> >> >
> > > >> >> > 2006/12/19, Vladimir Ivanov < ivavladimir@gmail.com>:
> > > >> >> > > On 12/19/06, Ivanov, Alexey A < alexey.a.ivanov@intel.com>
> > wrote:
> > > >> >> > >
> > > >> >> > > >
> > > >> >> > > > There's only one GUI test in your list:
> > > >> >> javax.swing.JToggleButtonTest.
> > > >> >> > > > The others test text model, and this particular
tests
> > don't use
> > > >> any
> > > >> >> > > > swing UI components at all.
> > > >> >> > > >
> > > >> >> > > > If I remember your reports correctly, the
latter three
> > tests
> > > >> fail
> > > >> >> > > > because of some serialization failure.
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > Yes, testParamString at javax.swing.JToggleButtonTest
and
> > > >> >> > testSerializable
> > > >> >> > > for other tests. But actually the stack trace is
similar
> > > >> (below) so
> > > >> I
> > > >> >> > think
> > > >> >> > > it not gui test problem. It is just reproduce this
issue.
> > > >> >> > >
> > > >> >> > >  Thanks, Vladimir
> > > >> >> > > Stack trace:
> > > >> >> > > Test: testParamStringClass: javax.swing.JToggleButtonTest
> > > >> >> > > java.lang.ArrayIndexOutOfBoundsException
> > > >> >> > > at java.util.Arrays.mergeSort(Arrays.java:2553)
> > > >> >> > > at java.util.Arrays.mergeSort (Arrays.java:2516)
> > > >> >> > > at java.util.Arrays.sort (Arrays.java:2872)
> > > >> >> > > at java.util.Arrays.sort(Arrays.java:2889)
> > > >> >> > > at java.beans.BeanInfoWrapper.getPropertyDescriptors
(
> > > >> >> > BeanInfoWrapper.java
> > > >> >> > > :77)
> > > >> >> > > at java.beans.BeanInfoWrapper.getPropertyDescriptors(
> > > >> >> > BeanInfoWrapper.java
> > > >> >> > > :74)
> > > >> >> > > at javax.swing.JComponent.paramString(JComponent.java:1334)
> > > >> >> > > at java.awt.Component.toString(Component.java:166)
> > > >> >> > > at
> > > >> >> javax.swing.JToggleButtonTest.testParamString(
> > JToggleButtonTest.java
> > > >> >> > :64)
> > > >> >> > > at
> > > >> >> java.lang.reflect.AccessibleObject.invokeV (
> > AccessibleObject.java:25)
> > > >> >> > > at
> > > >> >> javax.swing.BasicSwingTestCase.runBareSuper(
> > BasicSwingTestCase.java
> > > >> :117)
> > > >> >> > > at javax.swing.SwingTestCase$1.run(SwingTestCase.java:45)
> > > >> >> > > at
> > > >> >> java.awt.event.InvocationEvent.runAndNotify(InvocationEvent.java:92)
> > > >> >> > > at
> > > >> java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:81)
> > > >> >> > > at java.awt.EventQueueCore.dispatchEventImpl(
> > EventQueueCore.java
> > > >> :133)
> > > >> >> > > at java.awt.EventQueue.dispatchEvent(EventQueue.java:144)
> > > >> >> > > at
> > > >> >> java.awt.EventDispatchThread.runModalLoop (
> > EventDispatchThread.java:75)
> > > >> >> > > at java.awt.EventDispatchThread.run(EventDispatchThread.java
> > :48)
> > > >> >> > >
> > > >> >> > > Test: testSerializableClass:
> > > >> >> > > javax.swing.text.AbstractDocument_SerializationTest
> > > >> >> > > java.lang.ArrayIndexOutOfBoundsException
> > > >> >> > > at java.util.Arrays.mergeSort (Arrays.java:2553)
> > > >> >> > > at java.util.Arrays.mergeSort (Arrays.java:2516)
> > > >> >> > > at java.util.Arrays.mergeSort(Arrays.java:2517)
> > > >> >> > > at java.util.Arrays.sort(Arrays.java:2872)
> > > >> >> > > at java.util.Arrays.sort(Arrays.java:2889)
> > > >> >> > > at java.io.ObjectStreamClass.computeSerialVersionUID(
> > > >> >> > ObjectStreamClass.java
> > > >> >> > > :54)
> > > >> >> > > at
> > > >> java.io.ObjectStreamClass.addToCache(ObjectStreamClass.java:211)
> > > >> >> > > at java.io.ObjectStreamClass.lookupStreamClass(
> > > >> ObjectStreamClass.java
> > > >> >> > :937)
> > > >> >> > > at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java
> > :90)
> > > >> >> > > at java.io.ObjectStreamClass.addToCache (
> > ObjectStreamClass.java
> > > >> :23)
> > > >> >> > > at java.io.ObjectStreamClass.lookupStreamClass(
> > > >> ObjectStreamClass.java
> > > >> >> > :937)
> > > >> >> > > at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java
> > :90)
> > > >> >> > > at java.io.ObjectOutputStream.writeClassDescForClass
(
> > > >> >> > ObjectOutputStream.java
> > > >> >> > > :110)
> > > >> >> > > at java.io.ObjectOutputStream.writeNewObject(
> > > >> ObjectOutputStream.java
> > > >> >> > :1644)
> > > >> >> > > at java.io.ObjectOutputStream.writeObjectInternal
(
> > > >> >> > ObjectOutputStream.java
> > > >> >> > > :1956)
> > > >> >> > > at java.io.ObjectOutputStream.writeObject
> > > >> >> (ObjectOutputStream.java :1785)
> > > >> >> > > at
> > > >> >> java.io.ObjectOutputStream.writeObject (ObjectOutputStream.java
> > :1749)
> > > >> >> > > at javax.swing.BasicSwingTestCase.serializeObject
(
> > > >> >> > BasicSwingTestCase.java
> > > >> >> > > :496)
> > > >> >> > > at javax.swing.SerializableTestCase.setUp
> > > >> >> (SerializableTestCase.java :50)
> > > >> >> > > at javax.swing.text.AbstractDocument_SerializationTest.setUp
> > > >> >> > > (AbstractDocument_SerializationTest.java:43)
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > > Regards,
> > > >> >> > > > Alexey.
> > > >> >> > > >
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > >On 12/18/06, Geir Magnusson Jr. < geir@pobox.com>
wrote:
> > > >> >> > > > >>
> > > >> >> > > > >>
> > > >> >> > > > >>
> > > >> >> > > > >> Mikhail Loenko wrote:
> > > >> >> > > > >> > 2006/12/18, Geir Magnusson Jr.
< geir@pobox.com >:
> > > >> >> > > > >> >>
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> Mikhail Loenko wrote:
> > > >> >> > > > >> >> > 2006/12/1, Geir Magnusson
Jr. < geir@pobox.com>:
> > > >> >> > > > >> >> >>
> > > >> >> > > > >> >> >>
> > > >> >> > > > >> >> >> Mikhail Loenko
wrote:
> > > >> >> > > > >> >> >> > 4) We have
cruise controls running classlibrary
> > tests
> > > >> on
> > > >> >> > > > DRLVM.
> > > >> >> > > > >We
> > > >> >> > > > >> >> >> > need to decide
what will we do when
> > DRLVM+Classlib
> > > >> >> cruise
> > > >> >> > > > control
> > > >> >> > > > >> >> >> > reports failure.
> > > >> >> > > > >> >> >>
> > > >> >> > > > >> >> >> Stop and fix the
problem.  Is there really a
> > question
> > > >> >> > here?  I
> > > >> >> > > > >agree
> > > >> >> > > > >> >> >
> > > >> >> > > > >> >> > Yes, there is a question
here. "Stop and fix"
> > includes
> > > >> >> > > > "discuss".
> > > >> >> > > > >But
> > > >> >> > > > >>
> > > >> >> > > > >> >> > as we now know discussion
may take several days.
> > And
> > > >> while
> > > >> >> > some
> > > >> >> > > > >> people
> > > >> >> > > > >> >> > discuss what the problem
is other people can't
> > proceed
> > > >> with
> > > >> >> > > > >> >> > development and patch
> > > >> >> > > > >> >> > intagration.
> > > >> >> > > > >> >> >
> > > >> >> > > > >> >> > To have better pace
and better CC up-time we need
> > > >> something
> > > >> >> > else
> > > >> >> > > > but
> > > >> >> > > > >> >> not
> > > >> >> > > > >> >> > just "stop and fix".
I suggest "revert and
> > continue"
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> What's the difference, other
than debating the
> > > >> semantics of
> > > >> >> > "fix"
> > > >> >> > > > and
> > > >> >> > > > >> >> "revert"?
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> We all agree - but I still
don't think you're
> > clearly
> > > >> stating
> > > >> >> > the
> > > >> >> > > > >> >> problem.  I think that the
core problem is that we
> > don't
> > > >> >> > > > immediately
> > > >> >> > > > >> >> react to CC failure.
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> Immediately reacting to
CC failure should be the
> > first
> > > >> >> order of
> > > >> >> > > > the
> > > >> >> > > > >day
> > > >> >> > > > >> >> here.  Reacting to me is
making the decision,
> > quickly,
> > > >> about
> > > >> >> > > > either
> > > >> >> > > > >> >> rolling back the change
("reverting") or doing
> > something
> > > >> >> else.
> > > >> >> > > > The
> > > >> >> > > > >key
> > > >> >> > > > >>
> > > >> >> > > > >> >> is being responsive.
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> It seems that what happens
is that we wait, and then
> > sets
> > > >> of
> > > >> >> > > > changes
> > > >> >> > > > >> >> pile up, and I think that
doing mass rollbacks at
> > that
> > > >> point
> > > >> >> > will
> > > >> >> > > > >solve
> > > >> >> > > > >> >> it, but make a mess.
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> The example of what I envision
is when I broke the
> > > >> build in
> > > >> >> > DRLVM,
> > > >> >> > > > >> >> Gregory told me immediately,
and I fixed immediately
> > - w/o
> > > >> a
> > > >> >> > > > rollback.
> > > >> >> > > > >> >>
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> All I'm saying is :
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> 1) We need to be far better
with reaction time
> > > >> >> > > > >> >
> > > >> >> > > > >> > I would say we need to be far
better with
> > fixing/reverting
> > > >> >> time.
> > > >> >> > > > >> > If we reacted immediately and
than discussed for two
> > weeks
> > > >> >> -- we
> > > >> >> > > > would
> > > >> >> > > > >> not
> > > >> >> > > > >> > be better than where we are
now
> > > >> >> > > > >>
> > > >> >> > > > >> Yes, fixing/reverting is included.
It's what I meant.
> > > >> >> > > > >>
> > > >> >> > > > >> >
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> 2) We have intelligent people
- we can be agile in
> > this by
> > > >> >> > making
> > > >> >> > > > >> >> decisions (quickly!) on
a case by case basis what to
> > do.
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> I'll also suggest that we
ask each committer to
> > check the
> > > >> CC
> > > >> >> > event
> > > >> >> > > > >> >> stream before committing,
so you don't commit into a
> > bad
> > > >> >> state
> > > >> >> > of
> > > >> >> > > > >> things.
> > > >> >> > > > >> >>
> > > >> >> > > > >> >> One of my problems is that
I don't trust the CC
> > stream,
> > > >> and
> > > >> >> > don't
> > > >> >> > > > >> >> clearly see it because it's
mixed in the other drek
> > of the
> > > >> >> > > > commits@
> > > >> >> > > > >> list.
> > > >> >> > > > >> >
> > > >> >> > > > >> > The problem is intermittent
failures. I suggest that
> > we
> > > >> >> exclude
> > > >> >> > > > >graphics
> > > >> >> > > > >> > tests
> > > >> >> > > > >> > from CCs and probably have CC-specific
exclude lists
> > for
> > > >> >> > networking
> > > >> >> > > > >> tests
> > > >> >> > > > >> > (or fix all the known intermittent
failures right now
> > :)
> > > >> >> > > > >>
> > > >> >> > > > >> good idea - works for me.
> > > >> >> > > > >>
> > > >> >> > > > >> We need to drive into stability -
we've made amazing
> > progress
> > > >> in
> > > >> >> > the
> > > >> >> > > > >> last two months, and now we're down
to the really,
> > really
> > > >> hard
> > > >> >> > stuff.
> > > >> >> > > > I
> > > >> >> > > > >> think that excluding them to get
rock-solid CC
> > reporting is
> > > >> >> step 0,
> > > >> >> > > > >> and then step 1 is try and grind
out the intermittent
> > > >> failures.
> > > >> >> > > > >>
> > > >> >> > > > >> geir
> > > >> >> > > > >>
> > > >> >> > > > >>
> > > >> >> > > >
> > > >> >> > > > --
> > > >> >> > > > Alexey A. Ivanov
> > > >> >> > > > Intel Enterprise Solutions Software Division
> > > >> >> > > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >>
> > > >
> > >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message