Return-Path: 
 <dev-return-23614-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org>
Delivered-To: apmail-harmony-dev-archive@www.apache.org
Received: (qmail 29294 invoked from network); 29 Jan 2007 08:46:02 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 29 Jan 2007 08:46:02 -0000
Received: (qmail 33724 invoked by uid 500); 29 Jan 2007 08:46:04 -0000
Delivered-To: apmail-harmony-dev-archive@harmony.apache.org
Received: (qmail 33693 invoked by uid 500); 29 Jan 2007 08:46:04 -0000
Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@harmony.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@harmony.apache.org>
List-Post: <mailto:dev@harmony.apache.org>
List-Id: <dev.harmony.apache.org>
Reply-To: dev@harmony.apache.org
Delivered-To: mailing list dev@harmony.apache.org
Received: (qmail 33684 invoked by uid 99); 29 Jan 2007 08:46:03 -0000
Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jan 2007 00:46:03 -0800
X-ASF-Spam-Status: No, hits=2.0 required=10.0
	tests=HTML_MESSAGE,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (herse.apache.org: domain of ivavladimir@gmail.com
 designates 66.249.92.171 as permitted sender)
Received: from [66.249.92.171] (HELO ug-out-1314.google.com) (66.249.92.171)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jan 2007 00:45:54 -0800
Received: by ug-out-1314.google.com with SMTP id z36so992912uge
        for <dev@harmony.apache.org>; Mon, 29 Jan 2007 00:45:33 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=hBTFJk6OhaMAnYcho5r56GS7bV/bMwHOPoG7wffSnutkiWP4F9hxtUIiZ9unlUBcrcdIBZ8wjO2Sb9Drp6VOfVahlWpudclITWi21YQRMOXvrBJwfE7IGG9LiUk5v7cZmo3Ty9RmLbacR1+zd0sOrHRgN1xfD/akearcZT33F7M=
Received: by 10.78.204.20 with SMTP id b20mr4001408hug.1170060332678;
        Mon, 29 Jan 2007 00:45:32 -0800 (PST)
Received: by 10.78.145.6 with HTTP; Mon, 29 Jan 2007 00:45:32 -0800 (PST)
Message-ID: <7273946b0701290045i545922dh7c791d5c2b8eb763@mail.gmail.com>
Date: Mon, 29 Jan 2007 14:45:32 +0600
From: "Vladimir Ivanov" <ivavladimir@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [classlib][build] exclude list impl issues
In-Reply-To: <3FC8DDC5-6CFD-4311-A6DF-A1FBF1BCA8BF@pobox.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_63232_26341219.1170060332645"
References: <f944d83d0701160308u360558b5j18ceb821c0cddf74@mail.gmail.com>
	 <c3755b3a0701272329y16539678i90e046ea232bb63e@mail.gmail.com>
	 <636fd28e0701280159k2ccfb4b4l6054f345d542817f@mail.gmail.com>
	 <7273946b0701282028t37feafc1q8d81b797a9d4fa53@mail.gmail.com>
	 <3FC8DDC5-6CFD-4311-A6DF-A1FBF1BCA8BF@pobox.com>
X-Virus-Checked: Checked by ClamAV on apache.org

------=_Part_63232_26341219.1170060332645
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On 1/29/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
>
> On Jan 28, 2007, at 11:28 PM, Vladimir Ivanov wrote:
>
> > OK. Current changes for exclude lists were integrated and now we have
> > 3-level exclude list:
> >
> > exlude.common - tests, that failed over all platforms
> >
> > exclude.<platform> - tests, that failed over specific platform only
> >
> > exclude.<platform>.interm - tests that failed time to time over
> > specific
> > platform only.
>
> Quick q - why separate interm out?  why not just put in the platform
> file?


To do exclude lists clean up process easier. To delete test from the
platform list you should check that this test passed one time. To delete
test from 'intermittently failed' exclude list each test should be run in
cycle.

>
> > Any file in this chain may be skipped. Final exclude lists are
> > generated at
> > the build time and stored to the ${hy.hdk}/build directory. Thanks
> > to Alexei
> > Zakharov for this changes.
>
> Another quick q - why not just glom things together in memory?



If we will run tests against HDK it will nice to skip excluded tests. For
this reason exclude lists should be stored somewhere in the built product.

>
> >
> >
> > But I have one more question to discuss: should we use the tests from
> > 'intermittently failed' exclude list for preintegration testing?
>
> What does that mean?



OK. Now pre-integration test include 'ant test' with requirements that all
tests should be passed. While we have intermittently failed tests this
target sometimes report 'failed' status for these tests without correlation
with commit changes. We have 2 options here:

- exclude these tests in the platform-specific ex-list

- exclude these tests in the special ex-list

Each option has pro and contra: in the first case we can miss the regression
when intermittently failed test became always failed or delete test from
exclude lists if it passed only one time.

In the second case we have some overhead to support special exclude lists.

Note to detect regression the 'default' mode should be 'on (run)
intermittently failed tests' to test commit changes and 'off' to run tests
under CC.

I'll add switches to on/off levels for resulting exclude lists.

 thanks, Vladimir



> > If we use
> > it we may miss some regression when intermittently failed test will
> > failed
> > constantly however if we does not use it we need to run test twice
> > sometimes.
> > What is correct behavior?
> >
> >  thanks, Vladimir
> >
> >
> >
> > On 1/28/07, Alex Blewitt <alex.blewitt@gmail.com> wrote:
> >>
> >> Yeah, +1 for using common exclude lists. It makes it easier when
> >> Harmony gets ported to other operating systems. And I don't see the
> >> benefit of having empty lists in that case; and if nothing's failing,
> >> you don't need an empty list either :-)
> >>
> >> Alex.
> >>
> >> On 28/01/07, Alexey Petrenko <alexey.a.petrenko@gmail.com> wrote:
> >> > +1 from me for using common exclude lists and removing empty lists.
> >> >
> >> > SY, Alexey
> >> >
> >> > 2007/1/16, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> >> > > Folks,
> >> > >
> >> > > I've some concerns with recent updates for x-list support: can
> >> we use
> >> > > a common macros instead of copy-pasting 4 new targets into
> >> build.xml
> >> > > for each module?
> >> > > Or, if we can neglect creating a compiled x-file
> >> > > (${hy.hdk}/build/<module>.exclude), just use "if" attribute of
> >> > > <excludesfile>, like this:
> >> > >
> >> > > <available property="x.list.exist"
> >> > > file="exclude.${hy.platform}.${hy.test.vm.name}"/>
> >> > > ...
> >> > >             <batchtest>
> >> > >                 <fileset dir="${src.test.java}">
> >> > >                     <include name="**/*Test.java"/>
> >> > >                     <excludesfile name="exclude.common"/>
> >> > >                     <excludesfile name="${exclude.file} if="
> >> x.list.exist" />
> >> > >                 </fileset>
> >> > >             </batchtest>
> >> > >
> >> > > Also, I suggest to delete empty x-list remained after introducing
> >> common lists.
> >> > >
> >> > > Another issue is with "hy.test.vm.name", it was a surprise for
> >> me that
> >> > > it is not autodetected yet. Most obvious way to get it is to read
> >> > > "java.vm.name" property, this only requires running trivial test.
> >> > >
> >> > > --
> >> > > Alexey
> >> > >
> >> >
> >>
>
>

------=_Part_63232_26341219.1170060332645--