harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [performance] a few early benchmarks
Date Tue, 28 Nov 2006 15:48:04 GMT

Sergey Kuksenko wrote:
> On 11/28/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>> Mikhail Loenko wrote:
>> > And we need to make sure that a patch that improves DaCapo by 0.1%
>> > does not lower down the SPEC by 10%
>> Absolutely :)
>> >
>> > We need to agree on a target function to optimize, like some weighted
>> mean.
>> It would be useful to understand what SPECjbb is actually testing - can
>> someone describe it?
> First of all I wish to cite spec.org:
> *--------------------*
> *SPECjbb2005 (Java Server Benchmark)* is SPEC's benchmark for evaluating 
> the
> performance of server side Java. SPECjbb2005 evaluates the performance of
> server side Java by emulating a three-tier client/server system (with
> emphasis on the middle tier). The benchmark exercises the 
> implementations of
> the JVM (Java Virtual Machine), JIT (Just-In-Time) compiler, garbage
> collection, threads and some aspects of the operating system. 

No I/O?  Concurrency?

> It also
> measures the performance of CPUs, caches, memory hierarchy and the
> scalability of shared memory processors (SMPs). SPECjbb2005 provides a new
> enhanced workload, implemented in a more object-oriented manner to reflect
> how real-world applications are designed and introduces new features 
> such as
> XML processing and BigDecimal computations to make the benchmark a more
> realistic reflection of today's applications.

Any I/O?

> SPECjbb2005 Benchmark Highlights
>   - Emulates a 3-tier system, the most common type of server-side Java
>   application today.
>   - Business logic and object manipulation, the work of the middle tier,
>   predominate.
>   - Clients are replaced by driver threads, database storage by binary
>   trees of objects.

That means the clients are in the same VM.  No sockets or such?

>   - Increasing amounts of workload are applied, providing a graphical
>   view of scalability.
> -------------------------------------------<end of citation>
> Really, SPECjbb emulates a set of transactions to several warehouses like
> "order something, payment, stocks, etc.."
> SPECjbb measures performance sequentially on 1,2,3,.... threads that allows
> to evaluate scalability.

without I/O

> Also I may comment some details:
> - SPECjbb doesn't use any database. It's emulated using java.util API. Such
> implementation measures pure Java environment.

Ah.  No I/O

> - For example - java.math.BigDecimal class is de facto standard
> representation for financial calculations. Moreover, JDBC maps DECIMAL 
> (SQL)
> data to BigDecimal class. SPECjbb  uses BigDecimal extensively.
> - Also SPECjbb uses XML processing for emulation logging like it may be 
> done
> in real enterprise system.

Do you have an idea of what %-age of workload processing in the test 
suite is XML?


View raw message