harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naveen Neelakantam <neela...@uiuc.edu>
Subject Re: [testing][cruise-control] how testing infrastructure should be improved
Date Tue, 30 Jan 2007 17:31:49 GMT

On Jan 29, 2007, at 5:26 AM, Vladimir Ivanov wrote:

> On 1/26/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>>
>> > Created jira should be integrated to the buildtest  
>> infrastructure and
>> > buildtest module should be extended by different application
>> > scenario tests,
>> > reliability tests etc. In general case we will have a long CC cycle
>> > (up to 1
>> > week J ) and results of this runs should be processed with other
>> > procedure
>> > (for example, results should be upload to harmonytest.org and than
>> > jira
>> > issues should be created for all failures).
>>
>> Right - we should probably try to define what we mean by  "short",
>> "medium" and "long" scenarios, and I suggest:
>>
>> 1) what we have now (build and unit test drlvm, classlib) as the
>> "short" cycle for fast failure notification
>>
>> 2) test classlib w/ drlvm and J9  + the iterative test cycle as the
>> "medium" cycle
>>
>> 3) everything you talk about above as the "long" cycle
>
>
> OK. One minor note, some time ago on the dev@ list somebody ask to add
> 'server jit' mode. Is it OK to add it to the current CC ('short'  
> cycle)?

+1

Thanks for following up on this.

>
>>
>> >
>> > It requires some 'standard' interface for all integrated scripts. I
>> > like
>> > classlib interface so how about:
>> >  - call of "ant setup;ant" will run all available scripts;
>> >  - call of "ant -Dmodule=hit setup;ant" will run current version of
>> > CC –
>> > Harmony integration tests;
>> >  - call of "ant -Dmodule=eut setup;ant" will run Eclipse  unit test
>> > etc.
>> >
>>
>> > Note, in this case each module should implement proper 'setup'
>> > target and
>> > has configuration for CC. The root-script will iterate over all
>> > modules to
>> > call their 'setup' and this setup should include whole test setup
>> > (downloading software, adding modules cc-configuration to working
>> > configuration etc).
>> >
>> > Is it OK?
>>
>> I dunno - this sounds like disjoint and separate CC runs, rather than
>> a CC run with multiple projects.
>>
>> For example, I'd like to have a set of "modules", which would be
>> incomplete cc config files, that somehow get glommed into a bigger cc
>> file - maybe the config.xml would have some kind of include  
>> mechanism.
>>
>> Suppose that we have :
>>
>> trunk/cc/
>>     config.xml
>>     modules/
>>        default_module.xml
>>        hut_module.xml
>>        eut_module.xml
>>        iterative_module.xml
>>        dacapo_module.xml
>>        specjbb_module.xml
>>        short_module.xml
>>        medium_module.xml
>>        long_modules.xml
>>
>> so then I could do :
>>
>>   $ ant
>>
>> to get what we have now - runs the default module - or
>>
>>   $ ant -Dmodules=hut,eut,dacapo
>>
>> to run those...
>>
>> Something like "medium_module.xml" would look like  (the following is
>> 'psuedo-code') :
>>
>>     <includeconfig name="default_module.xml"/>
>>     <includeconfig name="dacapo_module.xml"/>
>>
>>
>> so that you can nest this as you want.
>>
>> >
>> > If nobody objects I'll start restructuring of buildtest module and
>> > will try
>> > to integrate one from extensions.
>>
>> Please describe how you want to do it.
>
>
> I think about following option: in the root file we have predefined  
> string.
> Something like 'modules=hut'. In the 'long' mode CC will iterate  
> over all
> entries. The medium cycle depend on users wishes and defined  
> through the
> command line like: 'ant -Dmodules=hut,iterative'. Each module has  
> predefined
> target (for example, 'setup'). In this target script should  
> download all
> software, apply all patches, add module configuration to the  
> current CC
> configuration (just copy content of the module configuration file  
> to the end
> of current configuration) etc. After 'ant <...> setup' we will have
> ready-to-run system with user-defined configuration.
>
>
>> >
>> >
>> > Thanks, Vladimir
>> >
>> >
>> >
>> > PS I think the resulting structure should be easy to extend and may
>> > looks
>> > like this:
>> >
>> > buildtest
>> >
>> > |--config  (default CC configuration to build classlib and DRLVM)
>> >
>> > |--hit (CC configuration to run Harmony classlib&DRLVM tests)
>> >
>> > |--eclipse
>> >
>> >     |-- eut (setup and CC configuration to run eclipse non- 
>> interactive
>> > tests)
>> >
>> >     |-- eclipse3.1.1
>> >
>> >         |-- some scenario
>> >
>> > |-- build.xml (common setup + call of module's 'setup')
>>
>>
>> Interesting.  I think one issue is that it seems like heavy lifting
>> to add a new module - each module becomes a "peer".   What do you
>> think of the other approach above?
>>
>> Either way, we don't want hit, eclipse, etc as peers.  If anything,
>> they should go in a modules/ directory...
>
>
>
> For each module we have at least 2 files: cc-config and build file.  
> But in
> some cases we will have some additional files (patches etc). For
> example, script for testing of JEdit application (issue 3012) has  
> about 15
> files. For me is better to store all staff in one place instead of  
> having
> parallel structure.
>
> thanks, Vladimir
>
> geir
>>
>>


Mime
View raw message