harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [general][repos] Benchmark repository
Date Wed, 16 Apr 2008 08:11:04 GMT
Stepan Mishura wrote:
> On 4/16/08, Alexey Varlamov <alexey.v.varlamov@gmail.com> wrote:
>> 2008/4/16, Stepan Mishura <stepan.mishura@gmail.com>:
>>> On 4/16/08, Alexey Varlamov <alexey.v.varlamov@gmail.com> wrote:
>>>> 2008/4/16, Tony Wu <wuyuehao@gmail.com>:
>>>>> Is it possible to integrate into BTI? if not we can consider the enhanced/tools/
>>>> There is one already in drlvm trunk, see working_vm/src/test/microbenchmark.
>>>> There is no infrastructure around, it is mere store holder for now.
>>>> Feel free to add benches there, they are not intended to be
>>>> VM-specific.
>>>>
>>> I would say opposite if they are DRLVM specific then it is OK to put
>>> them to the folder. Otherwise (i.e. they are not VM-specific) we
>>> should integrate them to BTI.
>>>
>>> Th point is that DRLVM workspace should contain only DRLVM specific
>>> tests. For example, IMO DRLVM workspace is not the right place for
>>> EHWA-API scenario.
>> This is too radical position IMO. Absolute majority of tests in DRLVM
>> are functional and not impl-specific, should we move them all to BTI?
>>
> 
> Yes, IMHO we should move them to BTI.
> 
> The point is that any workspace (classlib/drlvm/jdktools) is not a
> repository for a set of different suites.
> 
>> Sometimes convenience of using and extending is more important for
>> success. If we had appropriate infra for benchmarks I wouldn't argue,
>> but now I'm afraid most contributors would rather leave a bench-case
>> hanging in JIRA than dare to hack BTI. I'm happy to be proven wrong,
>> though.
>>
> 
> "convenience of using and extending" is questionable for me in this case.
> Well, yes I agree that from position of a DRLVM developer it is more
> convenient when EHWA-API scenario is located in DRLVM workspace - no
> additional efforts are required to run it. But what about classlib
> developer who wants to run EHWA-API on J9 - she/he needs to checkout
> DRLVM workspace. Is this convenient and extensible? (Hmm, may be I was
> wrong when I insisted on integration of LDAP scenario into BTI then
> into classlibrary ;-))
> 
> Seriously, if we think that using BTI is complicated for a developer
> then we should do our best and make it simpler and more convenient.
> Otherwise we finish with zoo of different suites/scenarios in several
> places.

I think you are right Stepan.  In fact it leads on to a bigger long term 
goal we should have to move all the tests over to the BTI, including the 
current classlib and VM-specific tests.  It will require us to define 
the new requirements on the BTI, such as the ability to run subsets of 
tests for classlib developers and JIT developers etc. as pre-commit 
testing.  A different, larger subset would then be the API tests that 
would be expected to run on any class library / VM combination.

In any case, putting the tests into DRLVM because BTI is too complex is 
a sign that it should be made simpler and more convenient, as you said.

Regards,
Tim

Mime
View raw message