harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Mazzocchi <stef...@apache.org>
Subject Re: [performance] a few early benchmarks
Date Wed, 29 Nov 2006 16:15:36 GMT
Sergey Kuksenko wrote:
> On 11/28/06, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>>
>>
>> >> It would be useful to understand what SPECjbb is actually testing -
>> can
>> >> someone describe it?
>> >
>> >
>> > First of all I wish to cite spec.org:
>> > *--------------------*
>> > *SPECjbb2005 (Java Server Benchmark)* is SPEC's benchmark for
>> evaluating
>> > the
>> > performance of server side Java. SPECjbb2005 evaluates the performance
>> of
>> > server side Java by emulating a three-tier client/server system (with
>> > emphasis on the middle tier). The benchmark exercises the
>> > implementations of
>> > the JVM (Java Virtual Machine), JIT (Just-In-Time) compiler, garbage
>> > collection, threads and some aspects of the operating system.
>>
>> No I/O?  Concurrency?
> 
> 
> No.
> 
>> It also
>> > measures the performance of CPUs, caches, memory hierarchy and the
>> > scalability of shared memory processors (SMPs). SPECjbb2005 provides a
>> new
>> > enhanced workload, implemented in a more object-oriented manner to
>> reflect
>> > how real-world applications are designed and introduces new features
>> > such as
>> > XML processing and BigDecimal computations to make the benchmark a more
>> > realistic reflection of today's applications.
>>
>> Any I/O?
>>
>> > SPECjbb2005 Benchmark Highlights
>> >
>> >   - Emulates a 3-tier system, the most common type of server-side Java
>> >   application today.
>> >   - Business logic and object manipulation, the work of the middle
>> tier,
>> >   predominate.
>> >   - Clients are replaced by driver threads, database storage by binary
>> >   trees of objects.
>>
>> That means the clients are in the same VM.  No sockets or such?
> 
> 
> You are right.
> No sockets, the clients are in the same VM.
> 
>>   - Increasing amounts of workload are applied, providing a graphical
>> >   view of scalability.
>> >
>> > -------------------------------------------<end of citation>
>> >
>> > Really, SPECjbb emulates a set of transactions to several warehouses
>> like
>> > "order something, payment, stocks, etc.."
>> >
>> > SPECjbb measures performance sequentially on 1,2,3,.... threads that
>> allows
>> > to evaluate scalability.
>>
>> without I/O
> 
> 
> yes.
> 
>>
>> > Also I may comment some details:
>> >
>> > - SPECjbb doesn't use any database. It's emulated using java.util API.
>> Such
>> > implementation measures pure Java environment.
>>
>> Ah.  No I/O
>>
>> >
>> > - For example - java.math.BigDecimal class is de facto standard
>> > representation for financial calculations. Moreover, JDBC maps DECIMAL
>> > (SQL)
>> > data to BigDecimal class. SPECjbb  uses BigDecimal extensively.
>> >
>> > - Also SPECjbb uses XML processing for emulation logging like it may be
>> > done
>> > in real enterprise system.
>>
>> Do you have an idea of what %-age of workload processing in the test
>> suite is XML?
>>
> 
> Below is SUN's (server) distribution on SPECjbb2005 shown using profiler.
> Garbage collection impact was not included into the data.
> Also the distribution below is shown accurately to methods inlining.
> 
> 
> 
> 
> impact
> 
> spec.jbb.* classes
> 
> 62.6%
> 
> java.lang.* classes
> 
> 9.2%
> 
> java.math.* classes
> 
> 6.8%
> 
> XML  classes
> 
> 6.7%
> 
> VM
> 
> 7.4%
> 
> java.util.* classes
> 
> 3.9%
> 
> UNKNOWN
> 
> 3.4%

and this is supposed to be "real life" workload for a server? without I/O?

You *have* to be kidding.

-- 
Stefano.


Mime
View raw message