harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [testing][cruise-control] how testing infrastructure should be improved
Date Tue, 30 Jan 2007 05:28:52 GMT
On 1/29/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
>
> > OK. One minor note, some time ago on the dev@ list somebody ask to add
> > 'server jit' mode. Is it OK to add it to the current CC ('short'
> > cycle)?
>
> Sure.  Why not.
>
> After staring at my two systems for a couple of days now, I've come
> to the conclusion that our short isn't short enough.  I think we need
> a "shorter" that simply builds stuff.  No tests.
>
> That would be quick "canary in the coalmine" test to help catch
> broken builds, before such breakage affects the longer running
> systems, because right now, the cycle is fairly long just for unit
> tests for drlvm and classlib
>
> geir


My timings are:
WinXP: classlib ~17 min, drlvm ~10, cl_tests ~34, drlvm_tests ~55
Linux ia32: ~15, ~11, ~40, ~50min
Linux em64t: ~11, ~11, ~120, ~50min

So, the cycle duration is about 2hours (I hope we can switch Linux em64t
from 'perTest' to 'once' mode soon :)).
If we define the 'short' cycle as 'build the classlib and the drlvm' the
minimal cycle will be reduced up to ~30 min. In this case we will have
'short' cycle ~30 min, user defined 'medium' cycle and all-included 'long'.
Is it OK?

 thanks, Vladimir



> >
> >>
> >> >
> >> > It requires some 'standard' interface for all integrated scripts. I
> >> > like
> >> > classlib interface so how about:
> >> >  - call of "ant setup;ant" will run all available scripts;
> >> >  - call of "ant -Dmodule=hit setup;ant" will run current version of
> >> > CC –
> >> > Harmony integration tests;
> >> >  - call of "ant -Dmodule=eut setup;ant" will run Eclipse  unit test
> >> > etc.
> >> >
> >>
> >> > Note, in this case each module should implement proper 'setup'
> >> > target and
> >> > has configuration for CC. The root-script will iterate over all
> >> > modules to
> >> > call their 'setup' and this setup should include whole test setup
> >> > (downloading software, adding modules cc-configuration to working
> >> > configuration etc).
> >> >
> >> > Is it OK?
> >>
> >> I dunno - this sounds like disjoint and separate CC runs, rather than
> >> a CC run with multiple projects.
> >>
> >> For example, I'd like to have a set of "modules", which would be
> >> incomplete cc config files, that somehow get glommed into a bigger cc
> >> file - maybe the config.xml would have some kind of include
> >> mechanism.
> >>
> >> Suppose that we have :
> >>
> >> trunk/cc/
> >>     config.xml
> >>     modules/
> >>        default_module.xml
> >>        hut_module.xml
> >>        eut_module.xml
> >>        iterative_module.xml
> >>        dacapo_module.xml
> >>        specjbb_module.xml
> >>        short_module.xml
> >>        medium_module.xml
> >>        long_modules.xml
> >>
> >> so then I could do :
> >>
> >>   $ ant
> >>
> >> to get what we have now - runs the default module - or
> >>
> >>   $ ant -Dmodules=hut,eut,dacapo
> >>
> >> to run those...
> >>
> >> Something like "medium_module.xml" would look like  (the following is
> >> 'psuedo-code') :
> >>
> >>     <includeconfig name="default_module.xml"/>
> >>     <includeconfig name="dacapo_module.xml"/>
> >>
> >>
> >> so that you can nest this as you want.
> >>
> >> >
> >> > If nobody objects I'll start restructuring of buildtest module and
> >> > will try
> >> > to integrate one from extensions.
> >>
> >> Please describe how you want to do it.
> >
> >
> > I think about following option: in the root file we have predefined
> > string.
> > Something like 'modules=hut'. In the 'long' mode CC will iterate
> > over all
> > entries. The medium cycle depend on users wishes and defined
> > through the
> > command line like: 'ant -Dmodules=hut,iterative'. Each module has
> > predefined
> > target (for example, 'setup'). In this target script should
> > download all
> > software, apply all patches, add module configuration to the
> > current CC
> > configuration (just copy content of the module configuration file
> > to the end
> > of current configuration) etc. After 'ant <...> setup' we will have
> > ready-to-run system with user-defined configuration.
> >
> >
> >> >
> >> >
> >> > Thanks, Vladimir
> >> >
> >> >
> >> >
> >> > PS I think the resulting structure should be easy to extend and may
> >> > looks
> >> > like this:
> >> >
> >> > buildtest
> >> >
> >> > |--config  (default CC configuration to build classlib and DRLVM)
> >> >
> >> > |--hit (CC configuration to run Harmony classlib&DRLVM tests)
> >> >
> >> > |--eclipse
> >> >
> >> >     |-- eut (setup and CC configuration to run eclipse non-
> >> interactive
> >> > tests)
> >> >
> >> >     |-- eclipse3.1.1
> >> >
> >> >         |-- some scenario
> >> >
> >> > |-- build.xml (common setup + call of module's 'setup')
> >>
> >>
> >> Interesting.  I think one issue is that it seems like heavy lifting
> >> to add a new module - each module becomes a "peer".   What do you
> >> think of the other approach above?
> >>
> >> Either way, we don't want hit, eclipse, etc as peers.  If anything,
> >> they should go in a modules/ directory...
> >
> >
> >
> > For each module we have at least 2 files: cc-config and build file.
> > But in
> > some cases we will have some additional files (patches etc). For
> > example, script for testing of JEdit application (issue 3012) has
> > about 15
> > files. For me is better to store all staff in one place instead of
> > having
> > parallel structure.
> >
> > thanks, Vladimir
> >
> > geir
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message