harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [drlvm] [testing] Excluding commit tests until the problem is fixed
Date Sat, 18 Nov 2006 15:34:48 GMT


Alexei Fedotov wrote:
> Do we have a best practice to link an exclusion to the reason? Are we
> intended to use svn blame for each line which contain <exclude>? Is it
> better to put comments right into the file near exludes? Thoughts?

What I meant by my comment is that I'm interested in seeing excludes in 
external files, which can then be organized by platform, and have that 
information used at the high-level in ant.  So for that existing JIRA, 
reusing code for managing exclude lists is a good idea, but I'm still 
fairly resistant to the idea of tying ourselves closer to junit.

As for info.... not a bad idea... yeah, it might be good to place info 
in comments in the exclude file, even if it's as simple as a JIRA entry 
id.  That way, if you want to see what's excluded, you can just look in 
that file, and "why" is there too...

geir

> 
> On 11/17/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>> I spoke too soon - do you mean reusing the code for managing the exclude
>> lists?
>>
>> geir
>>
>>
>> Geir Magnusson Jr. wrote:
>> >
>> >
>> > Tim Ellison wrote:
>> >> Before you go off writing more code, just take a moment to look at
>> >> HARMONY-263 and tell us what you think of it.
>> >
>> > It ties us to JUnit.  Doesn't moving the exclude list upwards give us
>> > more freedom?
>> >
>> > geir
>> >
>> >>
>> >> Thanks
>> >> Tim
>> >>
>> >> Alexei Zakharov wrote:
>> >>> Hi Vladimir,
>> >>>
>> >>> It seems everybody likes this approach. In that case, I have another
>> >>> idea for exclude lists. Can't we go further and extend the current
>> >>> exclude list functionality a bit more? And forget about TestNG and
>> >>> friends for a while I mean.
>> >>>
>> >>> For example, we can put exclude lists into something like:
>> >>>
>> >>> exclude.xml:
>> >>> ---
>> >>> <exclude-list>
>> >>>  <!-- exclude only particular tests -->
>> >>>  <class name="org.apache.harmony.luni.test.java.io.MyTest">
>> >>>    <test name="testConstructor11"/>
>> >>>    <test name="testMyMethodObjectObjectString_HY1234"/>
>> >>>  </class>
>> >>>  <!-- exclude all tests -->
>> >>>  <class name="org.apache.harmony.luni.test.java.io.NiceTest2"
>> >>> includeAll="true"/>
>> >>> ...
>> >>> </exclude-list>
>> >>>
>> >>> exclude.linux.drlvm.xml:
>> >>> ---
>> >>> <exclude-list>
>> >>>  <class name="org.apache.harmony.rmi.test.java.rmi.Ð’adBoyTest">
>> >>>    <test name="testLinuxHang_my"/>
>> >>>  </class>
>> >>> </exclude-list>
>> >>>
>> >>> And etc. ${hy.platfrom}and ${hy.harmony.vm.name} can be passed to the
>> >>> controller test suite by ant. By the controller test suite I mean the
>> >>> java class that knows how to parse the above files (using simple SAX
>> >>> parser for example - it is easy, I can help if needed) and implements
>> >>> junit TestSuite model to get fine-grained control over the testing
>> >>> process.
>> >>>
>> >>> IMHO this can be a nice solution for now. It's more powerful since it
>> >>> allows to exclude individual tests rather that whole classes. What do
>> >>> you think?
>> >>>
>> >>> Thanks,
>> >>>
>> >>>
>> >>> 2006/11/15, Vladimir Ivanov <ivavladimir@gmail.com>:
>> >>>> Seems, we says about different things :)
>> >>>>
>> >>>> First of all, we have no TestNG (or other harness) yet but we 
>> need now
>> >>>> different exclude lists for different platforms.
>> >>>>
>> >>>> Also, in my vision these exclude-lists are like a buffer before
we
>> >>>> mark test
>> >>>> by correct tags.
>> >>>> When the test fails on some platform we update the corresponding
>> >>>> x-list and
>> >>>> investigate this failure.
>> >>>> As the result of investigation we mark the test or fix it.
>> >>>>
>> >>>>  Thanks, Vladimir
>> >>>>
>> >>>>
>> >>>> On 11/15/06, Alexei Zakharov <alexei.zakharov@gmail.com> wrote:
>> >>>>> Things become more and more complicated. Can anyone say why
we
>> >>>>> rejected to use TestSuites for this purpose from the very 
>> beginning?
>> >>>>> Well, I can't say I am against using xml lists here. But the

>> next step
>> >>>>> will be to keep list of individual failing test methods in the
xml
>> >>>>> file. Then to create separate xml lists for api and impl tests

>> and so
>> >>>>> on. If we can't run original TestNG on Harmony then we invent
it by
>> >>>>> ourselves. :-)
>> >>>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> 2006/11/15, Vladimir Ivanov <ivavladimir@gmail.com>:
>> >>>>>> As part of solution for this issue the
>> >>>>>> 
>> *HARMONY-2197*<http://issues.apache.org/jira/browse/HARMONY-2197> was
>> >>>>>> created.
>> >>>>>>
>> >>>>>> I suggest using the separate exclude list for each platform.
I
>> >>>> hope in
>> >>>>> this
>> >>>>>> case the test enabling for the different platforms will
be easy.
>> >>>> Please,
>> >>>>>> look at it.
>> >>>>>>
>> >>>>>> Any comments are welcome :)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>  Thanks, Vladimir
>> >>>>>>
>> >>>>>>
>> >>>>>> On 11/15/06, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
>> >>>>>>> Pavel, you are correct. Rana, sorry for confusion. Both
issues
>> >>>> block
>> >>>>>>> passing class library unit tests.
>> >>>>>>>
>> >>>>>>> http://issues.apache.org/jira/browse/HARMONY-2070 [drlvm][thread]
>> >>>>>>> Unhandled exception in java.exe while java.util.jar
module tests
>> >>>>>>> execution
>> >>>>>>>
>> >>>>>>> http://issues.apache.org/jira/browse/HARMONY-2073 [drlvm][unit]
>> >>>>>>> org.apache.harmony.beans.tests.java.beans.PersistenceDelegateTest
>> >>>>>>>
>> >>>>>>> I've used a debugger and caught an assert in
>> >>>>>>> exn_raise_by_name_internal for the second one. The first
one
>> >>>> contains
>> >>>>>>> three diffrent issues, and I cannot say where exactly
the
>> >>>> problem is.
>> >>>>>>> On 11/15/06, Pavel Afremov < pavel.n.afremov@gmail.com>
wrote:
>> >>>>>>>> As I understand Alexey means HARMONY-2073, but not
HARMONY-2070.
>> >>>>>>>>
>> >>>>>>>> Alexei, is it correct? If not, could you clarify
the point about
>> >>>>>>>> exn_raise_by_name_internal in your initial letter,
please?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Pavel Afremov.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 11/8/06, Rana Dasgupta <rdasgupt@gmail.com>
wrote:
>> >>>>>>>>> OK thanks Pavel, I'll try the patch today.
>> >>>>>>>>>
>> >>>>>>>>> Rana
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On 11/8/06, Pavel Afremov <pavel.n.afremov@gmail.com>
wrote:
>> >>>>>>>>>> Hi Rana.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> I extend guard region as work around. It's
only one way,
>> >>>> which
>> >>>>> "fix"
>> >>>>>>> SOE
>> >>>>>>>>>> on
>> >>>>>>>>>> my SuSE Linux, without potential regression
of your fix.
>> >>>> On my
>> >>>>> Linux
>> >>>>>>>>>> machine
>> >>>>>>>>>> violation access signals happen one page
before protected
>> >>>> page
>> >>>>> on
>> >>>>>>> the
>> >>>>>>>>>> stack.
>> >>>>>>>>>> It's it.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> I ran all tests, and everything was OK.
But strange
>> >>>> misprint was
>> >>>>>>> fount
>> >>>>>>>>> in
>> >>>>>>>>>> the new test.
>> >>>>>>>>>>
>> >>>>>>>>>> So I attach new fixed patch.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Pavel Afremov.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On 11/8/06, Rana Dasgupta <rdasgupt@gmail.com>
wrote:
>> >>>>>>>>>>> Though I tried several times, I could
not repro 2070 or
>> >>>>> Alexey's
>> >>>>>>>>>> specific
>> >>>>>>>>>>> problems. The test attached to 2018
repros, and that I
>> >>>> think
>> >>>>> is
>> >>>>>>>>> enough.
>> >>>>>>>>>>> Pavel,
>> >>>>>>>>>>>   1. The patch looks good, but I could
not apply and try it
>> >>>>> since
>> >>>>>>> my
>> >>>>>>>>>> Linux
>> >>>>>>>>>>> box is down.
>> >>>>>>>>>>>   2. Did you run all tests ( smoke,
cuint, kernel, and
>> >>>>> classlib )?
>> >>>>>>>>> Since
>> >>>>>>>>>>> this fully turns on lazy exceptions,
we need to ensure that
>> >>>>> all
>> >>>>>>> tests
>> >>>>>>>>>>> pass,
>> >>>>>>>>>>> or at least have identical behaviour
before and after the
>> >>>>> pacth.
>> >>>>>>>>>>>   3. Adding a finalizer based stack
test to smoke is a good
>> >>>>> idea.
>> >>>>>>>>>>>   4. On Linux you extend the guard region
up ( or down
>> >>>>> whatever )
>> >>>>>>> by a
>> >>>>>>>>>>> page. Did you find a good reason for
it, or is this just
>> >>>> being
>> >>>>>>>>> careful?
>> >>>>>>>>>>> Rana
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On 11/7/06, Pavel Afremov < pavel.n.afremov@gmail.com>
>> >>>> wrote:
>> >>>>>>>>>>>> Rana,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Everything is correct in you description,
but it looks
>> >>>> like
>> >>>>> that
>> >>>>>>> *
>> >>>>>>>>>>>> HARMONY-2018* <
>> >>>>>>> https://issues.apache.org/jira/browse/HARMONY-2018>
>> >>>>>>>>>>> should
>> >>>>>>>>>>>> fix described bug. I think Alexei
will have a chance to
>> >>>>> check
>> >>>>>>> it.
>> >>>
>> >>>
>> >>
>> >
>>
> 
> 

Mime
View raw message