Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: pass (asf.osuosl.org: domain of t.p.ellison@gmail.com designates
 64.233.182.186 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding;
        b=jwpeH+pjb29cNiZtsVoU4l3dpQxZIqajG/5V6TtJD4FDwS8av8LdSnKknCqFzXU/r4zzmLTIf/YJEwBFgDwdQZoSoV4zXJPRrtOI040P7wBwbdtveIFzc1kPN5lvuv2i2N26XL3fNG4zJdVwSFnaVlGBXzqYBjBdJH3nYs4hteA=
Message-ID: <450A886D.3020104@gmail.com>
Date: Fri, 15 Sep 2006 12:03:09 +0100
From: Tim Ellison <t.p.ellison@gmail.com>
User-Agent: Thunderbird 1.5.0.5 (Windows/20060719)
MIME-Version: 1.0
To: harmony-dev@incubator.apache.org
Subject: [general] change the subject line...  (was: Re: [result] Re: [vote]
 HARMONY-1363 - DRLVM fixes and additions)
References: 
 <509223F0BF55E74FA1247D17207E7A0C65813E@orsmsx419.amr.corp.intel.com>
	 <4dd1f3f00609131409k310e4f56s594c927775202428@mail.gmail.com>
	 <uzmd3kp8k.fsf@gmail.com>
	 <51d555c70609132336j5b86e776lce5471d4fc3e9892@mail.gmail.com>
	 <469bff730609140141w666091c3xd80e7716ad684a60@mail.gmail.com>
	 <51d555c70609140716n142c2d38g597c9305582f2959@mail.gmail.com>
 <469bff730609150335l2bf52c20i42b25349ced06ce0@mail.gmail.com>
In-Reply-To: <469bff730609150335l2bf52c20i42b25349ced06ce0@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

at least, or better yet move to a new thread.  This conversation has
moved way beyond discussing the results of a contribution vote.

For the poor people who have hundreds of mails to read ;-)

Tim

Pavel Ozhdikhin wrote:
> Rana,
> 
> To cover most of the performance regressions ideally we might have three
> types of tests:
> 
>   1. High-level benchmarks (like SPECs or DaCapo benchmarks) - cover
>   significant performance issues or issues that are not related to a
>   particular optimization
>   2. Performance regression tests or micro-benchmarks - a
>   bytecode-based tests covering one optimization or a small group of them
>   3. Low-level tests (like VM or JIT unit tests) - tests for
>   particular code transformations.
> 
> To my mind the second type of tests is the most sensitive to the
> environment. In many cases it's difficult to create such test in a
> reasonable amount of time. I admit that this type of tests might be a
> short-term solution to check if a fix has been integrated properly but in
> the long-term we need also 1 and 3 from the list.
> 
> Thanks,
> Pavel
> 
> On 9/14/06, Rana Dasgupta <rdasgupt@gmail.com> wrote:
> 
>> Hi Pavel,
>>   Platform specific optimizations can be accomodated in the scheme
>> described by doing a cpuid check in the test and automatically passing it
>> or
>> disabling on all other platforms. That shouldn't be too hard.
>>   I understand that some jit optimizations are deeper and more abstract,
>> but ultimately the value of the optimization cannot just be the morphing
>> of
>> an IR, and the gain cannot be invisible to the user, or the regression
>> undetectable. If it needs to be part of a sequence to be effective, the
>> scenario in the test needs to be set up accordingly. It is a little
>> uncomfortable if a framework does some magic and then comes back and says
>> "everything is OK".
>>   Sorry to sound difficult.
>>
>> Thanks,
>> Rana
>>
>>
>> > On 9/14/06, Pavel Ozhdikhin <pavel.ozhdikhin@gmail.com> wrote:
>> > >
>> > > Hello Rana,
>> > >
>> > > When I think of an optimization which gives 1% improvement on some
>> > > simple
>> > > workload or 3% improvement on EM64T platforms only I doubt this
>> can be
>> > > easily detected with a general-purpose test suite. IMO the
>> performance
>> > > regression testing should have a specialized framework and a stable
>> > > environment which guarantees no user application can spoil the
>> results.
>> > >
>> > > The right solution might also be a JIT testing framework which would
>> > > understand the JIT IRs and check if some code patterns have been
>> > > optimized
>> > > as expected. Such way we can guarantee necessary optimizations are
>> done
>> > > independently of the user environment.
>> > >
>> > > Thanks,
>> > > Pavel
>> > >
>> > >
>> > >
>> > >
>> > >
>>
>>
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org