harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [performance] a few early benchmarks
Date Mon, 27 Nov 2006 05:24:16 GMT
2006/11/25, Stefano Mazzocchi <stefano@apache.org>:
> Alexey Petrenko wrote:
> > 23 Nov 2006 19:44:23 +0600, Egor Pasko <egor.pasko@gmail.com>:
> >> On the 0x22A day of Apache Harmony Alexey Petrenko wrote:
> >> > 2006/11/23, Mikhail Loenko <mloenko@gmail.com>:
> >> > > 23 Nov 2006 16:34:22 +0600, Egor Pasko <egor.pasko@gmail.com>:
> >> > > > On the 0x22A day of Apache Harmony Alexey Petrenko wrote:
> >> > > > > 2006/11/23, Vladimir Strigun <vstrigun@gmail.com>:
> >> > > > > > On 23 Nov 2006 14:37:09 +0600, Egor Pasko
> >> <egor.pasko@gmail.com> wrote:
> >> > > > > > > On the 0x22A day of Apache Harmony Vladimir Strigun
wrote:
> >> > > > > > > > The numbers that I published was received
on P4 under
> >> Windows +
> >> > > > > > > > server.emconf +Harmony-1980. Unfortunately
I haven't run
> >> Dacapo under
> >> > > > > > > > x86_64, but I hope we could receive almost
the same
> >> range (10-20 %
> >> > > > > > > > slower that Sun) with the mentioned configuration.
> >> > > > > > >
> >> > > > > > > And Sun was running with "-server" too I guess?
:)
> >> > > > > >
> >> > > > > > Yes, of course.
> >> > > > > >
> >> > > > > > > Maybe, it is time to track performance comparisons
of
> >> *different
> >> > > > > > > platforms* in one place? That should help to avoid
major
> >> differences in
> >> > > > > > > our visions for harmony performance.
> >> > > > > >
> >> > > > > > Yes, good idea. Should we define the configuration
for all
> >> VM's as well?
> >> > > > > > For instance, for Sun we could use parameters from
spec
> >> site. What do
> >> > > > > > you think about it?
> >> > > > > +1 for options from spec site.
> >> > > >
> >> > > > I am in love with it too. We could present results similar to
how
> >> > > > spec.org does. Just a list of runs.
> >> > > > For each:
> >> > > > * revision number
> >> > > > * hardware/os summary (number of cores)
> >> > > > * link to full details
> >> > > +1
> >> > >
> >> > > > * more?
> >> > >
> >> > > score? :)
> >>
> >> oops :)
> >>
> >> > Scores are most visible things but we can not easily publish them for
> >> > all specs...
> >> not allowed -> won't publish. We have great free benchmarks (thanks to
> >> DaCapo guys!), they will give us a good picture.
> > But I think that we should keep these (spec) benchmarks in mind and
> > optimize Harmony for them.
>
> Each and everyone of us is free to run harmony against anything they
> want and send info/patches that optimize for that.... but as a community
> we need to focus on what creates the best practical result.
>
> My proposal is to focus on DaCapo which is, more modern, more open, more
> complete, more thought out and more in line with real-life uses of the
> java technologies that apache users care for, today.
>
> And where DaCapo is lacking (for example, client/server heavily
> concurrent serving applications), we should work with them to add new tests.
>
> Spec on the other end is:
>
>  1) almost a decade old!
>  2) closed both sources, binaries and results
>  3) no community to talk to
>  4) no sign of activity
>
> There is the very high chance that the results of Spec will drive us to
> spend energy in fixing things that nobody cares about.
>
> Personally, I don't care about "brochure-like PR figures". Remember, our
> goal is not to sell harmony and convince people to adopt with figures...
> people will try it, they will run tests and if they find it good enough,
> they'll use it, if not they will use something else, or help us fix what
> they care for.
>
> So, if you care for SPEC, sure, bring it on, but it will be treated as
> just another application and the community will decide if those changes
> need to be fixed or not, depending on the 'short blanket' principle of
> parameter tuning.
>
> The official benchmark for Harmony must be something open, anything else
> is just another application.
>
> Because, believe me, it won't matter to anyone out there if Sun has 2x
> performance against spec when harmony sends 2x more servlet requests ;-)

Not sure that when we elect an official benchmark we should take
into account whether it's open or not. IMHO we should base our
decision on how close it to the real life and how many people are
interested in it

Thanks,
Mikhail


>
> --
> Stefano.
>
>

Mime
View raw message