harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Mazzocchi <stef...@apache.org>
Subject Re: [performance] a few early benchmarks
Date Fri, 24 Nov 2006 21:01:29 GMT
Alexey Petrenko wrote:
> 23 Nov 2006 19:44:23 +0600, Egor Pasko <egor.pasko@gmail.com>:
>> On the 0x22A day of Apache Harmony Alexey Petrenko wrote:
>> > 2006/11/23, Mikhail Loenko <mloenko@gmail.com>:
>> > > 23 Nov 2006 16:34:22 +0600, Egor Pasko <egor.pasko@gmail.com>:
>> > > > On the 0x22A day of Apache Harmony Alexey Petrenko wrote:
>> > > > > 2006/11/23, Vladimir Strigun <vstrigun@gmail.com>:
>> > > > > > On 23 Nov 2006 14:37:09 +0600, Egor Pasko
>> <egor.pasko@gmail.com> wrote:
>> > > > > > > On the 0x22A day of Apache Harmony Vladimir Strigun
wrote:
>> > > > > > > > The numbers that I published was received on P4
under
>> Windows +
>> > > > > > > > server.emconf +Harmony-1980. Unfortunately I haven't
run
>> Dacapo under
>> > > > > > > > x86_64, but I hope we could receive almost the
same
>> range (10-20 %
>> > > > > > > > slower that Sun) with the mentioned configuration.
>> > > > > > >
>> > > > > > > And Sun was running with "-server" too I guess? :)
>> > > > > >
>> > > > > > Yes, of course.
>> > > > > >
>> > > > > > > Maybe, it is time to track performance comparisons
of
>> *different
>> > > > > > > platforms* in one place? That should help to avoid
major
>> differences in
>> > > > > > > our visions for harmony performance.
>> > > > > >
>> > > > > > Yes, good idea. Should we define the configuration for all
>> VM's as well?
>> > > > > > For instance, for Sun we could use parameters from spec
>> site. What do
>> > > > > > you think about it?
>> > > > > +1 for options from spec site.
>> > > >
>> > > > I am in love with it too. We could present results similar to how
>> > > > spec.org does. Just a list of runs.
>> > > > For each:
>> > > > * revision number
>> > > > * hardware/os summary (number of cores)
>> > > > * link to full details
>> > > +1
>> > >
>> > > > * more?
>> > >
>> > > score? :)
>>
>> oops :)
>>
>> > Scores are most visible things but we can not easily publish them for
>> > all specs...
>> not allowed -> won't publish. We have great free benchmarks (thanks to
>> DaCapo guys!), they will give us a good picture.
> But I think that we should keep these (spec) benchmarks in mind and
> optimize Harmony for them.

Each and everyone of us is free to run harmony against anything they
want and send info/patches that optimize for that.... but as a community
we need to focus on what creates the best practical result.

My proposal is to focus on DaCapo which is, more modern, more open, more
complete, more thought out and more in line with real-life uses of the
java technologies that apache users care for, today.

And where DaCapo is lacking (for example, client/server heavily
concurrent serving applications), we should work with them to add new tests.

Spec on the other end is:

 1) almost a decade old!
 2) closed both sources, binaries and results
 3) no community to talk to
 4) no sign of activity

There is the very high chance that the results of Spec will drive us to
spend energy in fixing things that nobody cares about.

Personally, I don't care about "brochure-like PR figures". Remember, our
goal is not to sell harmony and convince people to adopt with figures...
people will try it, they will run tests and if they find it good enough,
they'll use it, if not they will use something else, or help us fix what
they care for.

So, if you care for SPEC, sure, bring it on, but it will be treated as
just another application and the community will decide if those changes
need to be fixed or not, depending on the 'short blanket' principle of
parameter tuning.

The official benchmark for Harmony must be something open, anything else
is just another application.

Because, believe me, it won't matter to anyone out there if Sun has 2x
performance against spec when harmony sends 2x more servlet requests ;-)

-- 
Stefano.


Mime
View raw message