harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Egor Pasko <egor.pa...@gmail.com>
Subject Re: [testing][cruise-control] how testing infrastructure should be improved
Date Tue, 30 Jan 2007 06:16:27 GMT
On the 0x26D day of Apache Harmony Vladimir Ivanov wrote:
> On 1/26/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
> >
> >
> > > Created jira should be integrated to the buildtest infrastructure and
> > > buildtest module should be extended by different application
> > > scenario tests,
> > > reliability tests etc. In general case we will have a long CC cycle
> > > (up to 1
> > > week J ) and results of this runs should be processed with other
> > > procedure
> > > (for example, results should be upload to harmonytest.org and than
> > > jira
> > > issues should be created for all failures).
> >
> > Right - we should probably try to define what we mean by  "short",
> > "medium" and "long" scenarios, and I suggest:
> >
> > 1) what we have now (build and unit test drlvm, classlib) as the
> > "short" cycle for fast failure notification
> >
> > 2) test classlib w/ drlvm and J9  + the iterative test cycle as the
> > "medium" cycle
> >
> > 3) everything you talk about above as the "long" cycle
> 
> 
> OK. One minor note, some time ago on the dev@ list somebody ask to add
> 'server jit' mode. Is it OK to add it to the current CC ('short' cycle)?

sure, you can even set the -Xem:server to exercise _instead_ of
-Xem:opt in CC. 'server' mode checks JIT more extensively in
most cases (except some corners that are not likely to catch a bug)

> >
> > >
> > > It requires some 'standard' interface for all integrated scripts. I
> > > like
> > > classlib interface so how about:
> > >  - call of "ant setup;ant" will run all available scripts;
> > >  - call of "ant -Dmodule=hit setup;ant" will run current version of
> > > CC √
> > > Harmony integration tests;
> > >  - call of "ant -Dmodule=eut setup;ant" will run Eclipse  unit test
> > > etc.
> > >
> >
> > > Note, in this case each module should implement proper 'setup'
> > > target and
> > > has configuration for CC. The root-script will iterate over all
> > > modules to
> > > call their 'setup' and this setup should include whole test setup
> > > (downloading software, adding modules cc-configuration to working
> > > configuration etc).
> > >
> > > Is it OK?
> >
> > I dunno - this sounds like disjoint and separate CC runs, rather than
> > a CC run with multiple projects.
> >
> > For example, I'd like to have a set of "modules", which would be
> > incomplete cc config files, that somehow get glommed into a bigger cc
> > file - maybe the config.xml would have some kind of include mechanism.
> >
> > Suppose that we have :
> >
> > trunk/cc/
> >     config.xml
> >     modules/
> >        default_module.xml
> >        hut_module.xml
> >        eut_module.xml
> >        iterative_module.xml
> >        dacapo_module.xml
> >        specjbb_module.xml
> >        short_module.xml
> >        medium_module.xml
> >        long_modules.xml
> >
> > so then I could do :
> >
> >   $ ant
> >
> > to get what we have now - runs the default module - or
> >
> >   $ ant -Dmodules=hut,eut,dacapo
> >
> > to run those...
> >
> > Something like "medium_module.xml" would look like  (the following is
> > 'psuedo-code') :
> >
> >     <includeconfig name="default_module.xml"/>
> >     <includeconfig name="dacapo_module.xml"/>
> >
> >
> > so that you can nest this as you want.
> >
> > >
> > > If nobody objects I'll start restructuring of buildtest module and
> > > will try
> > > to integrate one from extensions.
> >
> > Please describe how you want to do it.
> 
> 
> I think about following option: in the root file we have predefined string.
> Something like 'modules=hut'. In the 'long' mode CC will iterate over all
> entries. The medium cycle depend on users wishes and defined through the
> command line like: 'ant -Dmodules=hut,iterative'. Each module has predefined
> target (for example, 'setup'). In this target script should download all
> software, apply all patches, add module configuration to the current CC
> configuration (just copy content of the module configuration file to the end
> of current configuration) etc. After 'ant <...> setup' we will have
> ready-to-run system with user-defined configuration.
> 
> 
> > >
> > >
> > > Thanks, Vladimir
> > >
> > >
> > >
> > > PS I think the resulting structure should be easy to extend and may
> > > looks
> > > like this:
> > >
> > > buildtest
> > >
> > > |--config  (default CC configuration to build classlib and DRLVM)
> > >
> > > |--hit (CC configuration to run Harmony classlib&DRLVM tests)
> > >
> > > |--eclipse
> > >
> > >     |-- eut (setup and CC configuration to run eclipse non-interactive
> > > tests)
> > >
> > >     |-- eclipse3.1.1
> > >
> > >         |-- some scenario
> > >
> > > |-- build.xml (common setup + call of module's 'setup')
> >
> >
> > Interesting.  I think one issue is that it seems like heavy lifting
> > to add a new module - each module becomes a "peer".   What do you
> > think of the other approach above?
> >
> > Either way, we don't want hit, eclipse, etc as peers.  If anything,
> > they should go in a modules/ directory...
> 
> 
> 
> For each module we have at least 2 files: cc-config and build file. But in
> some cases we will have some additional files (patches etc). For
> example, script for testing of JEdit application (issue 3012) has about 15
> files. For me is better to store all staff in one place instead of having
> parallel structure.
> 
>  thanks, Vladimir
> 
> geir
> >
> >

-- 
Egor Pasko


Mime
View raw message