harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivanov, Alexey A" <alexey.a.iva...@intel.com>
Subject RE: [general] aiming no regression
Date Tue, 19 Dec 2006 07:37:04 GMT
>-----Original Message-----
>From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
>Sent: Tuesday, December 19, 2006 7:48 AM
>To: dev@harmony.apache.org; geir@pobox.com
>Subject: Re: [general] aiming no regression
>
>I collect some statistics based on CC notifications for period
December,
>8-18.
>For the claslib runs over IBMVM and DRLVM we have:
>
>
>
>Regression detected, notifications: 6
>
>Intermittent zero-lengh reports on IBMVM (usually, swing): 4
>
>Intermittent failure of swing tests over IBMVM: 5
>
>Intermittent failure of swing tests over DRLVM: 1
>
>Intermittent failure of network tests: 9
>
>Intermittent failure of nio tests: 4
>
>
>
>Detailed data for classlib tests:
>
>
>
>Failures of swing tests (IBMVM on Windows only):
>
>javax.swing.JToggleButtonTest         3
>
>javax.swing.text.AbstractDocument_SerializationTest           5
>
>javax.swing.text.DefaultStyledDocumentTest            5
>
>javax.swing.text.GapContent_SerializeTest               2
>
>
>
>Failures of nio tests (IBMVM and DRLVM but Linux only):
>
>org.apache.harmony.nio.tests.java.nio.channels.DatagramChannelTest
>2
>
>org.apache.harmony.nio.tests.java.nio.channels.ServerSocketChannelTest
>1
>
>org.apache.harmony.nio.tests.java.nio.channels.SocketChannelTest  1
>
>
>
>Failures of net tests:
>
>tests.api.java.net.ConnectExceptionTest      1
>
>tests.api.java.net.ServerSocketTest              1
>
>HttpURLConnectionTest                              7
>
>
>
>Current status:
>
>1) HttpURLConnectionTest test was excluded (thanks to Tim)
>
>2) Issue 2438 updated by ex-list to exclude: 4 swing test. Wait for
>integration
>
>
>
>So, to increase stability of CC we need to investigate 5 tests (3 nio +
2
>net) and do a big work to enable intermittently failed GUI tests.
>

There's only one GUI test in your list: javax.swing.JToggleButtonTest.
The others test text model, and this particular tests don't use any
swing UI components at all.

If I remember your reports correctly, the latter three tests fail
because of some serialization failure.

Regards,
Alexey.

>
>
>On 12/18/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>>
>>
>> Mikhail Loenko wrote:
>> > 2006/12/18, Geir Magnusson Jr. <geir@pobox.com >:
>> >>
>> >>
>> >> Mikhail Loenko wrote:
>> >> > 2006/12/1, Geir Magnusson Jr. <geir@pobox.com>:
>> >> >>
>> >> >>
>> >> >> Mikhail Loenko wrote:
>> >> >> > 4) We have cruise controls running classlibrary tests on
DRLVM.
>We
>> >> >> > need to decide what will we do when DRLVM+Classlib cruise
control
>> >> >> > reports failure.
>> >> >>
>> >> >> Stop and fix the problem.  Is there really a question here?  I
>agree
>> >> >
>> >> > Yes, there is a question here. "Stop and fix" includes
"discuss".
>But
>>
>> >> > as we now know discussion may take several days. And while some
>> people
>> >> > discuss what the problem is other people can't proceed with
>> >> > development and patch
>> >> > intagration.
>> >> >
>> >> > To have better pace and better CC up-time we need something else
but
>> >> not
>> >> > just "stop and fix". I suggest "revert and continue"
>> >>
>> >> What's the difference, other than debating the semantics of "fix"
and
>> >> "revert"?
>> >>
>> >> We all agree - but I still don't think you're clearly stating the
>> >> problem.  I think that the core problem is that we don't
immediately
>> >> react to CC failure.
>> >>
>> >> Immediately reacting to CC failure should be the first order of
the
>day
>> >> here.  Reacting to me is making the decision, quickly, about
either
>> >> rolling back the change ("reverting") or doing something else.
The
>key
>>
>> >> is being responsive.
>> >>
>> >> It seems that what happens is that we wait, and then sets of
changes
>> >> pile up, and I think that doing mass rollbacks at that point will
>solve
>> >> it, but make a mess.
>> >>
>> >> The example of what I envision is when I broke the build in DRLVM,
>> >> Gregory told me immediately, and I fixed immediately - w/o a
rollback.
>> >>
>> >>
>> >> All I'm saying is :
>> >>
>> >> 1) We need to be far better with reaction time
>> >
>> > I would say we need to be far better with fixing/reverting time.
>> > If we reacted immediately and than discussed for two weeks -- we
would
>> not
>> > be better than where we are now
>>
>> Yes, fixing/reverting is included. It's what I meant.
>>
>> >
>> >>
>> >> 2) We have intelligent people - we can be agile in this by making
>> >> decisions (quickly!) on a case by case basis what to do.
>> >>
>> >> I'll also suggest that we ask each committer to check the CC event
>> >> stream before committing, so you don't commit into a bad state of
>> things.
>> >>
>> >> One of my problems is that I don't trust the CC stream, and don't
>> >> clearly see it because it's mixed in the other drek of the
commits@
>> list.
>> >
>> > The problem is intermittent failures. I suggest that we exclude
>graphics
>> > tests
>> > from CCs and probably have CC-specific exclude lists for networking
>> tests
>> > (or fix all the known intermittent failures right now :)
>>
>> good idea - works for me.
>>
>> We need to drive into stability - we've made amazing progress in the
>> last two months, and now we're down to the really, really hard stuff.
I
>> think that excluding them to get rock-solid CC reporting is step 0,
>> and then step 1 is try and grind out the intermittent failures.
>>
>> geir
>>
>>

--
Alexey A. Ivanov
Intel Enterprise Solutions Software Division

Mime
View raw message