Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of ivavladimir@gmail.com
 designates 66.249.92.174 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=KMvDjmPlSIj3qSnUyoIJEIXOrCMh273Ld/Q4KTKXqGA9jHXaZDJoG6fRM497gqF0tEmImKrDV7xmCsC9cXzKOCdEj24zXYI1BHUcS/d4NAFmXPirBiplV9S3SzU9fOEu///wAwhPH7RPKqBkKEWFyuT4duM0DWSSGXAjjMzBblY=
Message-ID: <7273946b0701290326s34054e21ve97d91d28264438c@mail.gmail.com>
Date: Mon, 29 Jan 2007 17:26:37 +0600
From: "Vladimir Ivanov" <ivavladimir@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [testing][cruise-control] how testing infrastructure should be
 improved
In-Reply-To: <C908AD10-5B89-4FCE-AE43-AFFB12AEFC3F@pobox.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_64439_24653394.1170069997655"
References: <7273946b0701260350x7ba25a8eu4303028a13188133@mail.gmail.com>
	 <C908AD10-5B89-4FCE-AE43-AFFB12AEFC3F@pobox.com>

------=_Part_64439_24653394.1170069997655
Content-Type: text/plain; charset=WINDOWS-1252; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On 1/26/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
>
> > Created jira should be integrated to the buildtest infrastructure and
> > buildtest module should be extended by different application
> > scenario tests,
> > reliability tests etc. In general case we will have a long CC cycle
> > (up to 1
> > week J ) and results of this runs should be processed with other
> > procedure
> > (for example, results should be upload to harmonytest.org and than
> > jira
> > issues should be created for all failures).
>
> Right - we should probably try to define what we mean by  "short",
> "medium" and "long" scenarios, and I suggest:
>
> 1) what we have now (build and unit test drlvm, classlib) as the
> "short" cycle for fast failure notification
>
> 2) test classlib w/ drlvm and J9  + the iterative test cycle as the
> "medium" cycle
>
> 3) everything you talk about above as the "long" cycle


OK. One minor note, some time ago on the dev@ list somebody ask to add
'server jit' mode. Is it OK to add it to the current CC ('short' cycle)?

>
> >
> > It requires some 'standard' interface for all integrated scripts. I
> > like
> > classlib interface so how about:
> >  - call of "ant setup;ant" will run all available scripts;
> >  - call of "ant -Dmodule=3Dhit setup;ant" will run current version of
> > CC =96
> > Harmony integration tests;
> >  - call of "ant -Dmodule=3Deut setup;ant" will run Eclipse  unit test
> > etc.
> >
>
> > Note, in this case each module should implement proper 'setup'
> > target and
> > has configuration for CC. The root-script will iterate over all
> > modules to
> > call their 'setup' and this setup should include whole test setup
> > (downloading software, adding modules cc-configuration to working
> > configuration etc).
> >
> > Is it OK?
>
> I dunno - this sounds like disjoint and separate CC runs, rather than
> a CC run with multiple projects.
>
> For example, I'd like to have a set of "modules", which would be
> incomplete cc config files, that somehow get glommed into a bigger cc
> file - maybe the config.xml would have some kind of include mechanism.
>
> Suppose that we have :
>
> trunk/cc/
>     config.xml
>     modules/
>        default_module.xml
>        hut_module.xml
>        eut_module.xml
>        iterative_module.xml
>        dacapo_module.xml
>        specjbb_module.xml
>        short_module.xml
>        medium_module.xml
>        long_modules.xml
>
> so then I could do :
>
>   $ ant
>
> to get what we have now - runs the default module - or
>
>   $ ant -Dmodules=3Dhut,eut,dacapo
>
> to run those...
>
> Something like "medium_module.xml" would look like  (the following is
> 'psuedo-code') :
>
>     <includeconfig name=3D"default_module.xml"/>
>     <includeconfig name=3D"dacapo_module.xml"/>
>
>
> so that you can nest this as you want.
>
> >
> > If nobody objects I'll start restructuring of buildtest module and
> > will try
> > to integrate one from extensions.
>
> Please describe how you want to do it.


I think about following option: in the root file we have predefined string.
Something like 'modules=3Dhut'. In the 'long' mode CC will iterate over all
entries. The medium cycle depend on users wishes and defined through the
command line like: 'ant -Dmodules=3Dhut,iterative'. Each module has predefi=
ned
target (for example, 'setup'). In this target script should download all
software, apply all patches, add module configuration to the current CC
configuration (just copy content of the module configuration file to the en=
d
of current configuration) etc. After 'ant <...> setup' we will have
ready-to-run system with user-defined configuration.


> >
> >
> > Thanks, Vladimir
> >
> >
> >
> > PS I think the resulting structure should be easy to extend and may
> > looks
> > like this:
> >
> > buildtest
> >
> > |--config  (default CC configuration to build classlib and DRLVM)
> >
> > |--hit (CC configuration to run Harmony classlib&DRLVM tests)
> >
> > |--eclipse
> >
> >     |-- eut (setup and CC configuration to run eclipse non-interactive
> > tests)
> >
> >     |-- eclipse3.1.1
> >
> >         |-- some scenario
> >
> > |-- build.xml (common setup + call of module's 'setup')
>
>
> Interesting.  I think one issue is that it seems like heavy lifting
> to add a new module - each module becomes a "peer".   What do you
> think of the other approach above?
>
> Either way, we don't want hit, eclipse, etc as peers.  If anything,
> they should go in a modules/ directory...


For each module we have at least 2 files: cc-config and build file. But in
some cases we will have some additional files (patches etc). For
example, script for testing of JEdit application (issue 3012) has about 15
files. For me is better to store all staff in one place instead of having
parallel structure.

 thanks, Vladimir

geir
>
>

------=_Part_64439_24653394.1170069997655--