Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: local policy)
Mime-Version: 1.0 (Apple Message framework v752.3)
In-Reply-To: <7273946b0701300417t58c12ce4q430906862126ea2c@mail.gmail.com>
References: <7273946b0701260350x7ba25a8eu4303028a13188133@mail.gmail.com>
 <C908AD10-5B89-4FCE-AE43-AFFB12AEFC3F@pobox.com>
 <7273946b0701290326s34054e21ve97d91d28264438c@mail.gmail.com>
 <22BCA896-5258-42F0-90AB-AF2F5A280FF2@pobox.com>
 <7273946b0701292128v474f96b2hf1a92f367ecae40a@mail.gmail.com>
 <AC88E17B-4516-4E1D-82B1-1C90A2902A2A@pobox.com>
 <7273946b0701300417t58c12ce4q430906862126ea2c@mail.gmail.com>
Content-Type: text/plain; charset=WINDOWS-1252; delsp=yes; format=flowed
Message-Id: <232248F6-5BCB-4442-90D9-65320DCA509F@uiuc.edu>
Content-Transfer-Encoding: quoted-printable
From: Naveen Neelakantam <neelakan@uiuc.edu>
Subject: Re: [testing][cruise-control] how testing infrastructure should be
 improved
Date: Tue, 30 Jan 2007 11:25:55 -0600
To: dev@harmony.apache.org


On Jan 30, 2007, at 6:17 AM, Vladimir Ivanov wrote:

> On 1/30/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>>
>> > So, the cycle duration is about 2hours (I hope we can switch Linux
>> > em64t
>> > from 'perTest' to 'once' mode soon :)).
>>
>> I'm running x86_64 (remember, we decided to use that token rather
>> than em64t :) in 'once' mode - I didn't ahve to switch to perTest.
>
>
>
> That really good news :) I'll try it tomorrow on my SUSE 9.

However, RHEL4 still is running CC in perTest.  I tried running it in =20=

"once" mode, and I started seeing OutOfMemory errors.

>
>> If we define the 'short' cycle as 'build the classlib and the
>> > drlvm' the
>> > minimal cycle will be reduced up to ~30 min. In this case we =20
>> will have
>> > 'short' cycle ~30 min, user defined 'medium' cycle and all-included
>> > 'long'.
>> > Is it OK?
>>
>> I think so.  I think we need better names that are descriptive rather
>> than relative, so we can add more w/o resorting to things like "ultra
>> medium short long" or whatever...
>>
>> so maybe :
>>
>> "build-profile" : just build classlib, drlvm and tools
>> "unit_profile" : runs the drlvm tests and classlib tests, can be used
>> after "build-profile"
>> "interative_profile" : can run after either the above, does iterative
>> testing
>> "functional1_profile" : ...
>> etc
>>
>> ?
>>
>> I don't know CC well enough - can I have these pre-set things, and
>> tell CC to do "build-profile" only, or do build-profile, and then if
>> that passes, do the unit-profile?
>
>
> Seems, that I also don't know CC as well as I want :(  As I =20
> understand it,
> CC operates with projects (in CC terms). Each project may depend on =20=

> other
> project status. For example, in the current buildtest configuration =20=

> drlvm
> build depends on classlib build status, run of drlvm tests depends =20
> on drlvm
> build status and run of classlib tests depends on classlib and =20
> drlvm builds
> status. So this dependency defined on the project level. In the =20
> case of
> different independent profiles seems them it will depends on =20
> classlib and
> drlvm builds only.
>
>
>> geir
>>
>> >
>> > thanks, Vladimir
>> >
>> >
>> >
>> >> >
>> >> >>
>> >> >> >
>> >> >> > It requires some 'standard' interface for all integrated
>> >> scripts. I
>> >> >> > like
>> >> >> > classlib interface so how about:
>> >> >> >  - call of "ant setup;ant" will run all available scripts;
>> >> >> >  - call of "ant -Dmodule=3Dhit setup;ant" will run current
>> >> version of
>> >> >> > CC =96
>> >> >> > Harmony integration tests;
>> >> >> >  - call of "ant -Dmodule=3Deut setup;ant" will run Eclipse
>> >> unit test
>> >> >> > etc.
>> >> >> >
>> >> >>
>> >> >> > Note, in this case each module should implement proper =20
>> 'setup'
>> >> >> > target and
>> >> >> > has configuration for CC. The root-script will iterate =20
>> over all
>> >> >> > modules to
>> >> >> > call their 'setup' and this setup should include whole test
>> >> setup
>> >> >> > (downloading software, adding modules cc-configuration to
>> >> working
>> >> >> > configuration etc).
>> >> >> >
>> >> >> > Is it OK?
>> >> >>
>> >> >> I dunno - this sounds like disjoint and separate CC runs,
>> >> rather than
>> >> >> a CC run with multiple projects.
>> >> >>
>> >> >> For example, I'd like to have a set of "modules", which =20
>> would be
>> >> >> incomplete cc config files, that somehow get glommed into a
>> >> bigger cc
>> >> >> file - maybe the config.xml would have some kind of include
>> >> >> mechanism.
>> >> >>
>> >> >> Suppose that we have :
>> >> >>
>> >> >> trunk/cc/
>> >> >>     config.xml
>> >> >>     modules/
>> >> >>        default_module.xml
>> >> >>        hut_module.xml
>> >> >>        eut_module.xml
>> >> >>        iterative_module.xml
>> >> >>        dacapo_module.xml
>> >> >>        specjbb_module.xml
>> >> >>        short_module.xml
>> >> >>        medium_module.xml
>> >> >>        long_modules.xml
>> >> >>
>> >> >> so then I could do :
>> >> >>
>> >> >>   $ ant
>> >> >>
>> >> >> to get what we have now - runs the default module - or
>> >> >>
>> >> >>   $ ant -Dmodules=3Dhut,eut,dacapo
>> >> >>
>> >> >> to run those...
>> >> >>
>> >> >> Something like "medium_module.xml" would look like  (the
>> >> following is
>> >> >> 'psuedo-code') :
>> >> >>
>> >> >>     <includeconfig name=3D"default_module.xml"/>
>> >> >>     <includeconfig name=3D"dacapo_module.xml"/>
>> >> >>
>> >> >>
>> >> >> so that you can nest this as you want.
>> >> >>
>> >> >> >
>> >> >> > If nobody objects I'll start restructuring of buildtest
>> >> module and
>> >> >> > will try
>> >> >> > to integrate one from extensions.
>> >> >>
>> >> >> Please describe how you want to do it.
>> >> >
>> >> >
>> >> > I think about following option: in the root file we have =20
>> predefined
>> >> > string.
>> >> > Something like 'modules=3Dhut'. In the 'long' mode CC will =
iterate
>> >> > over all
>> >> > entries. The medium cycle depend on users wishes and defined
>> >> > through the
>> >> > command line like: 'ant -Dmodules=3Dhut,iterative'. Each module =20=

>> has
>> >> > predefined
>> >> > target (for example, 'setup'). In this target script should
>> >> > download all
>> >> > software, apply all patches, add module configuration to the
>> >> > current CC
>> >> > configuration (just copy content of the module configuration =20
>> file
>> >> > to the end
>> >> > of current configuration) etc. After 'ant <...> setup' we =20
>> will have
>> >> > ready-to-run system with user-defined configuration.
>> >> >
>> >> >
>> >> >> >
>> >> >> >
>> >> >> > Thanks, Vladimir
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > PS I think the resulting structure should be easy to extend
>> >> and may
>> >> >> > looks
>> >> >> > like this:
>> >> >> >
>> >> >> > buildtest
>> >> >> >
>> >> >> > |--config  (default CC configuration to build classlib and
>> >> DRLVM)
>> >> >> >
>> >> >> > |--hit (CC configuration to run Harmony classlib&DRLVM tests)
>> >> >> >
>> >> >> > |--eclipse
>> >> >> >
>> >> >> >     |-- eut (setup and CC configuration to run eclipse non-
>> >> >> interactive
>> >> >> > tests)
>> >> >> >
>> >> >> >     |-- eclipse3.1.1
>> >> >> >
>> >> >> >         |-- some scenario
>> >> >> >
>> >> >> > |-- build.xml (common setup + call of module's 'setup')
>> >> >>
>> >> >>
>> >> >> Interesting.  I think one issue is that it seems like heavy
>> >> lifting
>> >> >> to add a new module - each module becomes a "peer".   What =20
>> do you
>> >> >> think of the other approach above?
>> >> >>
>> >> >> Either way, we don't want hit, eclipse, etc as peers.  If
>> >> anything,
>> >> >> they should go in a modules/ directory...
>> >> >
>> >> >
>> >> >
>> >> > For each module we have at least 2 files: cc-config and build =20=

>> file.
>> >> > But in
>> >> > some cases we will have some additional files (patches etc). For
>> >> > example, script for testing of JEdit application (issue 3012) =20=

>> has
>> >> > about 15
>> >> > files. For me is better to store all staff in one place =20
>> instead of
>> >> > having
>> >> > parallel structure.
>> >> >
>> >> > thanks, Vladimir
>> >> >
>> >> > geir
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>