harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [drlvm] [testing] Excluding commit tests until the problem is fixed
Date Fri, 17 Nov 2006 09:57:36 GMT
Hi Vladimir,

It seems everybody likes this approach. In that case, I have another
idea for exclude lists. Can't we go further and extend the current
exclude list functionality a bit more? And forget about TestNG and
friends for a while I mean.

For example, we can put exclude lists into something like:

exclude.xml:
---
<exclude-list>
  <!-- exclude only particular tests -->
  <class name="org.apache.harmony.luni.test.java.io.MyTest">
    <test name="testConstructor11"/>
    <test name="testMyMethodObjectObjectString_HY1234"/>
  </class>
  <!-- exclude all tests -->
  <class name="org.apache.harmony.luni.test.java.io.NiceTest2"
includeAll="true"/>
...
</exclude-list>

exclude.linux.drlvm.xml:
---
<exclude-list>
  <class name="org.apache.harmony.rmi.test.java.rmi.Ð’adBoyTest">
    <test name="testLinuxHang_my"/>
  </class>
</exclude-list>

And etc. ${hy.platfrom}and ${hy.harmony.vm.name} can be passed to the
controller test suite by ant. By the controller test suite I mean the
java class that knows how to parse the above files (using simple SAX
parser for example - it is easy, I can help if needed) and implements
junit TestSuite model to get fine-grained control over the testing
process.

IMHO this can be a nice solution for now. It's more powerful since it
allows to exclude individual tests rather that whole classes. What do
you think?

Thanks,


2006/11/15, Vladimir Ivanov <ivavladimir@gmail.com>:
> Seems, we says about different things :)
>
> First of all, we have no TestNG (or other harness) yet but we need now
> different exclude lists for different platforms.
>
> Also, in my vision these exclude-lists are like a buffer before we mark test
> by correct tags.
> When the test fails on some platform we update the corresponding x-list and
> investigate this failure.
> As the result of investigation we mark the test or fix it.
>
>  Thanks, Vladimir
>
>
> On 11/15/06, Alexei Zakharov <alexei.zakharov@gmail.com> wrote:
> >
> > Things become more and more complicated. Can anyone say why we
> > rejected to use TestSuites for this purpose from the very beginning?
> > Well, I can't say I am against using xml lists here. But the next step
> > will be to keep list of individual failing test methods in the xml
> > file. Then to create separate xml lists for api and impl tests and so
> > on. If we can't run original TestNG on Harmony then we invent it by
> > ourselves. :-)
> >
> > Thanks,
> >
> > 2006/11/15, Vladimir Ivanov <ivavladimir@gmail.com>:
> > > As part of solution for this issue the
> > > *HARMONY-2197*<http://issues.apache.org/jira/browse/HARMONY-2197> was
> > > created.
> > >
> > > I suggest using the separate exclude list for each platform. I hope in
> > this
> > > case the test enabling for the different platforms will be easy. Please,
> > > look at it.
> > >
> > > Any comments are welcome :)
> > >
> > >
> > >
> > >  Thanks, Vladimir
> > >
> > >
> > > On 11/15/06, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > > >
> > > > Pavel, you are correct. Rana, sorry for confusion. Both issues block
> > > > passing class library unit tests.
> > > >
> > > > http://issues.apache.org/jira/browse/HARMONY-2070 [drlvm][thread]
> > > > Unhandled exception in java.exe while java.util.jar module tests
> > > > execution
> > > >
> > > > http://issues.apache.org/jira/browse/HARMONY-2073 [drlvm][unit]
> > > > org.apache.harmony.beans.tests.java.beans.PersistenceDelegateTest
> > > >
> > > > I've used a debugger and caught an assert in
> > > > exn_raise_by_name_internal for the second one. The first one contains
> > > > three diffrent issues, and I cannot say where exactly the problem is.
> > > >
> > > > On 11/15/06, Pavel Afremov < pavel.n.afremov@gmail.com> wrote:
> > > > > As I understand Alexey means HARMONY-2073, but not HARMONY-2070.
> > > > >
> > > > > Alexei, is it correct? If not, could you clarify the point about
> > > > > exn_raise_by_name_internal in your initial letter, please?
> > > > >
> > > > >
> > > > > Pavel Afremov.
> > > > >
> > > > >
> > > > > On 11/8/06, Rana Dasgupta <rdasgupt@gmail.com> wrote:
> > > > > >
> > > > > > OK thanks Pavel, I'll try the patch today.
> > > > > >
> > > > > > Rana
> > > > > >
> > > > > >
> > > > > > On 11/8/06, Pavel Afremov <pavel.n.afremov@gmail.com>
wrote:
> > > > > > >
> > > > > > > Hi Rana.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I extend guard region as work around. It's only one way,
which
> > "fix"
> > > > SOE
> > > > > > > on
> > > > > > > my SuSE Linux, without potential regression of your fix.
On my
> > Linux
> > > >
> > > > > > > machine
> > > > > > > violation access signals happen one page before protected
page
> > on
> > > > the
> > > > > > > stack.
> > > > > > > It's it.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I ran all tests, and everything was OK. But strange misprint
was
> > > > fount
> > > > > > in
> > > > > > > the new test.
> > > > > > >
> > > > > > > So I attach new fixed patch.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Pavel Afremov.
> > > > > > >
> > > > > > >
> > > > > > > On 11/8/06, Rana Dasgupta <rdasgupt@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Though I tried several times, I could not repro 2070
or
> > Alexey's
> > > > > > > specific
> > > > > > > > problems. The test attached to 2018 repros, and that
I think
> > is
> > > > > > enough.
> > > > > > > >
> > > > > > > > Pavel,
> > > > > > > >   1. The patch looks good, but I could not apply and
try it
> > since
> > > > my
> > > > > > > Linux
> > > > > > > > box is down.
> > > > > > > >   2. Did you run all tests ( smoke, cuint, kernel,
and
> > classlib )?
> > > >
> > > > > > Since
> > > > > > > > this fully turns on lazy exceptions, we need to ensure
that
> > all
> > > > tests
> > > > > > > > pass,
> > > > > > > > or at least have identical behaviour before and after
the
> > pacth.
> > > > > > > >   3. Adding a finalizer based stack test to smoke
is a good
> > idea.
> > > > > > > >   4. On Linux you extend the guard region up ( or
down
> > whatever )
> > > > by a
> > > > > > > > page. Did you find a good reason for it, or is this
just being
> > > > > > careful?
> > > > > > > >
> > > > > > > > Rana
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 11/7/06, Pavel Afremov < pavel.n.afremov@gmail.com>
wrote:
> > > > > > > > >
> > > > > > > > > Rana,
> > > > > > > > >
> > > > > > > > > Everything is correct in you description, but
it looks like
> > that
> > > > *
> > > > > > > > > HARMONY-2018* <
> > > > https://issues.apache.org/jira/browse/HARMONY-2018>
> > > > > > > > should
> > > > > > > > > fix described bug. I think Alexei will have a
chance to
> > check
> > > > it.



-- 
Alexei Zakharov,
Intel Enterprise Solutions Software Division
Mime
View raw message