2007/9/13, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
>
> Stepan,
>
> IMO current way of snapshot results publishing is a workaround
> basically, due to lack of adequate support in BTI to the moment. We
> should not hack snapshot adapter each time new suite/adapter is added
> or changed.
> So we need to extend BTI for more flexible and powerfull publishing
> facilities in some time (presumably after M3 competion), and then get
> rid of that custom publishers in snapshot.
>
> OTOH I agree, the way Sean suggests is too specific; I'd prefer keep
> in BTI most general functionalities.
> So there could be 2 adaptors, 1st instruments JRE and constitutes a
> "super-suite" for tests like Stepan suggested, and 2nd depends on all
> tests and generates summary report in the end.


 It's OK for me.
Then i will try to implement these two adaptors if no one object.


A couple of issues lies
> on this way, though:
> 1) How to configure emma for each suite, leaving an adaptor of that
> suite intact? I.e. which kernel-classes-list to take, where to store
> coverage report, etc - looks like Sean has some answers or ideas here;
> anyway there should not be more than light massaging of emma code.
> Gathering total cumulative coverage report is enough, I think.


Leo has found a way to get kernel -classes-list for us as he mentioned
before, thanks, Leo.
I suggest put the coverage report in one place rather than in many places.

2) Currently BTI handles suites dependency strictly, always skipping
> dependent suite run if it's super-suite has failed. So if some test
> fails, coverage adapter won't be run. We may just add customization to
> BTI to enable simple run ordering vs strong dependency.


If we put the coverage meta info into a centralized place like
${root}/build/emma.
We can just run the 2nd adaptor along, since it's responsebility is just to
generate the report
according to the centralized emma meta info if any . There is no need to
depend on any other adaptors.
Is this approach all right?

Anyway, i agree with the suggestion.



> --
> Alexey
>
>
> 2007/9/13, Stepan Mishura <stepan.mishura@gmail.com>:
> > On 9/13/07, Sean Qiu wrote:
> > > We all know the scenario for emma is :
> > > 1) instument jre
> > > 2) run test within the instumented jre and generate coverage file
> > > 3) collect the coverage file and generate the coverage report
> > >
> > > For further study of emma, i have two concerns:
> > > 1. We have to maintain this task outside the adaptor, since it need to
> run
> > > the (3) step after test.
> > >    This cannot be achieved elegantly by a single adaptor but add a new
> > > target in main script like "run-cc"
> > >     Agree? Commnets?
> > >
> > > 2. If we donot change the existing adaptor , the coverage file will be
> > > placed in the dir where test is invoked.
> > >    I think we should locate them in a fixed place at least, like
> > > build/results/${test_suites}.
> > >    A alternative solution is put them all in a stand along dir, like
> > > build/emma
> > >    For both of them, we need add a new line in existing adaptor to
> tell
> > > emma where we want to place our coverage file.
> > >    Is this modification acceptable?
> > >
> > > If nobody object, i will start to:
> > > 1) Add the emma target in main script
> > > 2) Try to put coverage file into a stand along dir
> > >
> > > Any suggestions are welcomed.
> > >
> >
> > I think that no changes to the framework or existing adaptors are
> > needed. Collection of coverage files and report generation for each
> > suite can by done by emma publisher (like snapshot testing does).
> >
> > So I'd implement emma adaptor in the way similar to the snapshot
> > testing, for example:
> >
> > 1) emma setup target - downloads and unpacks emma
> > 2) emma run target - instruments jre
> >
> > 3) emma adaptor depends on jre to be instrumented (it may be snapshot,
> > drlvm or hdk project)
> > (i.e. emma.parameters.depends=(snapshot|drlvm|hdk))
> >
> > 4) all suites depend on emma project.
> > ( <suite>.parameters.required.tested.runtime=${
> emma.parameters.shared.instrumented.jre})
> >
> > 5) create custom config.xml for CC with emma publisher for suites
> > (<suite>..parameters.cc.config=${adaptors.dir}/emma/custom-config.xml)
> >
> > Thanks,
> > Stepan.
> >
>



-- 
Sean Qiu
China Software Development Lab, IBM