harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [general] aiming no regression
Date Tue, 19 Dec 2006 04:48:09 GMT
I collect some statistics based on CC notifications for period December,
8-18.
For the claslib runs over IBMVM and DRLVM we have:



Regression detected, notifications: 6

Intermittent zero-lengh reports on IBMVM (usually, swing): 4

Intermittent failure of swing tests over IBMVM: 5

Intermittent failure of swing tests over DRLVM: 1

Intermittent failure of network tests: 9

Intermittent failure of nio tests: 4



Detailed data for classlib tests:



Failures of swing tests (IBMVM on Windows only):

javax.swing.JToggleButtonTest         3

javax.swing.text.AbstractDocument_SerializationTest           5

javax.swing.text.DefaultStyledDocumentTest            5

javax.swing.text.GapContent_SerializeTest               2



Failures of nio tests (IBMVM and DRLVM but Linux only):

org.apache.harmony.nio.tests.java.nio.channels.DatagramChannelTest
2

org.apache.harmony.nio.tests.java.nio.channels.ServerSocketChannelTest     1

org.apache.harmony.nio.tests.java.nio.channels.SocketChannelTest  1



Failures of net tests:

tests.api.java.net.ConnectExceptionTest      1

tests.api.java.net.ServerSocketTest              1

HttpURLConnectionTest                              7



Current status:

1) HttpURLConnectionTest test was excluded (thanks to Tim)

2) Issue 2438 updated by ex-list to exclude: 4 swing test. Wait for
integration



So, to increase stability of CC we need to investigate 5 tests (3 nio + 2
net) and do a big work to enable intermittently failed GUI tests.



On 12/18/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
>
>
> Mikhail Loenko wrote:
> > 2006/12/18, Geir Magnusson Jr. <geir@pobox.com >:
> >>
> >>
> >> Mikhail Loenko wrote:
> >> > 2006/12/1, Geir Magnusson Jr. <geir@pobox.com>:
> >> >>
> >> >>
> >> >> Mikhail Loenko wrote:
> >> >> > 4) We have cruise controls running classlibrary tests on DRLVM.
We
> >> >> > need to decide what will we do when DRLVM+Classlib cruise control
> >> >> > reports failure.
> >> >>
> >> >> Stop and fix the problem.  Is there really a question here?  I agree
> >> >
> >> > Yes, there is a question here. "Stop and fix" includes "discuss". But
>
> >> > as we now know discussion may take several days. And while some
> people
> >> > discuss what the problem is other people can't proceed with
> >> > development and patch
> >> > intagration.
> >> >
> >> > To have better pace and better CC up-time we need something else but
> >> not
> >> > just "stop and fix". I suggest "revert and continue"
> >>
> >> What's the difference, other than debating the semantics of "fix" and
> >> "revert"?
> >>
> >> We all agree - but I still don't think you're clearly stating the
> >> problem.  I think that the core problem is that we don't immediately
> >> react to CC failure.
> >>
> >> Immediately reacting to CC failure should be the first order of the day
> >> here.  Reacting to me is making the decision, quickly, about either
> >> rolling back the change ("reverting") or doing something else.  The key
>
> >> is being responsive.
> >>
> >> It seems that what happens is that we wait, and then sets of changes
> >> pile up, and I think that doing mass rollbacks at that point will solve
> >> it, but make a mess.
> >>
> >> The example of what I envision is when I broke the build in DRLVM,
> >> Gregory told me immediately, and I fixed immediately - w/o a rollback.
> >>
> >>
> >> All I'm saying is :
> >>
> >> 1) We need to be far better with reaction time
> >
> > I would say we need to be far better with fixing/reverting time.
> > If we reacted immediately and than discussed for two weeks -- we would
> not
> > be better than where we are now
>
> Yes, fixing/reverting is included. It's what I meant.
>
> >
> >>
> >> 2) We have intelligent people - we can be agile in this by making
> >> decisions (quickly!) on a case by case basis what to do.
> >>
> >> I'll also suggest that we ask each committer to check the CC event
> >> stream before committing, so you don't commit into a bad state of
> things.
> >>
> >> One of my problems is that I don't trust the CC stream, and don't
> >> clearly see it because it's mixed in the other drek of the commits@
> list.
> >
> > The problem is intermittent failures. I suggest that we exclude graphics
> > tests
> > from CCs and probably have CC-specific exclude lists for networking
> tests
> > (or fix all the known intermittent failures right now :)
>
> good idea - works for me.
>
> We need to drive into stability - we've made amazing progress in the
> last two months, and now we're down to the really, really hard stuff.  I
> think that excluding them to get rock-solid CC reporting is step 0,
> and then step 1 is try and grind out the intermittent failures.
>
> geir
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message