openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Linskey" <plins...@gmail.com>
Subject Re: Comparison Metrics for OpenJPA's ConcurrentHashMap
Date Mon, 04 Jun 2007 23:11:08 GMT
Incidentally, I'm working on code that allows for toggling of
concurrent map providers in different contexts. We could probably
adapt this to be more general-purpose. However, currently, this is not
something that can happen via our current configuration mechanisms,
since some of the key maps in question are constructed before any
configuration bootstrapping happens.

-Patrick

On 6/4/07, Dain Sundstrom <dain@iq80.com> wrote:
> I think this thread shows that there are some critical maps at the
> core of JPA, and it shows that the optimal map for an environment is
> debatable.  Is this something we could make configurable?  I'm
> personally very very skeptical of micro-benchmarks, and won't be
> convinced which is best until I see a full application benchmark
> (that is representative of my application).
>
> -dain
>
> On Jun 4, 2007, at 5:34 AM, David Ezzio wrote:
>
> > Two
> >
> > Patrick Linskey wrote:
> >> Hi,
> >>
> >> How many cores / CPUs were you using in these tests?
> >>
> >> -Patrick
> >>
> >> On 6/1/07, David Ezzio (asmtp) <dezzio@bea.com> wrote:
> >>> Hi Marc,
> >>>
> >>> I still was not able to get HighScaleLib to work. It produces a
> >>> SecurityException when attempting to get the Unsafe object. I
> >>> decided to
> >>> avoid changing the relevant security policy.
> >>>
> >>> On the other hand, I did test Emory University's backport of the
> >>> java.util.concurrent package. This provides to Java 1.4 a
> >>> ConcurrentHashMap implementation that is compatible with the one
> >>> found
> >>> in Java 5 and 6.
> >>>
> >>> I also realized that I could get another metric from my numbers, the
> >>> percentage of time the threads were suspended during the metered
> >>> operations. Then I reran the tests for Emory's backport using fewer
> >>> threads. In the classic pattern of an overloaded CPU, the higher
> >>> thread
> >>> count both lowers throughput and increases response time.
> >>> (Throughput
> >>> is yet another number I haven't extracted from the data, although
> >>> it is
> >>> obvious when running the tests.)
> >>>
> >>> All of the previous tests were run with 5 writing and 10 reading
> >>> threads. Only backport was additionally tested with 2 writing and 4
> >>> reading threads. The suspended percentage is approximate since the
> >>> adding and updating tests have slightly different numbers.
> >>>
> >>> Implementation  Add   Remove  Update  Find-a  Find-r  Find-u
> >>> Suspended
> >>> --------------+------+-------+-------+-------+-------+-------
> >>> +---------
> >>> synchronized     103     35      50      40      37     54
> >>> concurrent      13.2    6.4     6.1     0.6     0.3    1.1
> >>> OpenJPA         29.8   26.6    27.9     0.6     0.6    0.6
> >>> Backport (5/10) 43.2   36.8    40.4     0.3     0.2    0.5       62%
> >>> Backport (2/4)   6.1    3.1     3.3     0.3     0.2    0.3        4%
> >>>
> >>> David
> >>>
> >>>
> >>> David Ezzio (asmtp) wrote:
> >>>> Hi Marc,
> >>>>
> >>>> I did plug it in, but it failed straightaway on a security
> >>>> issue.  I
> >>>> should probably read its documentation. :)  I'll try it again along
> >>> with
> >>>> the backport lib done by Emory U.
> >>>>
> >>>> David
> >>>>
> >>>>
> >>>> Marc Prud'hommeaux wrote:
> >>>>> David-
> >>>>>
> >>>>> That is very interesting.
> >>>>>
> >>>>> Did you also take a look at the one at
> >>>>> http://sourceforge.net/projects/high-scale-lib ? They say its
> >>>>> performance only shines for high thread/cpu counts, but it
> >>>>> might be
> >>>>> interesting to see where its numbers lie in the range.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On May 29, 2007, at 11:01 AM, David Ezzio (asmtp) wrote:
> >>>>>
> >>>>>> Recently, I did some testing of Map implementations under
> >>> concurrency.
> >>>>>>
> >>>>>> My primary purpose was to verify the reliability of OpenJPA's
> >>>>>> ConcurrentHashMap implementation. As I got into it, I saw the
> >>>>>> opportunity to get some performance metrics out of the test.
> >>>>>>
> >>>>>> The biggest part of my task was coming up with a reliable and
> >>>>>> useful
> >>>>>> testing framework. I design it with the following two factors
in
> >>>>>> mind: First, I wanted to test the edge conditions where an
> >>>>>> entry had
> >>>>>> just been added or removed or where a key's value had just been
> >>>>>> updated. The idea is that a number of threads add, remove, and
> >>>>>> update
> >>>>>> entries, while other threads check to see if these recent
> >>>>>> modifications are visible (or in the case of removals, not
> >>>>>> visible).
> >>>>>> Second, I wanted the testing framework itself to be free of
> >>>>>> synchronization. If the testing framework used synchronization
> >>>>>> then
> >>>>>> it would tend to serialize the readers and writers and thereby
> >>>>>> mask
> >>>>>> concurrency issues in the map implementation under test.
> >>>>>>
> >>>>>> The testing framework uses a non-synchronizing, non-blocking
FIFO
> >>>>>> queue as the mechanism for the writing threads to communicate
> >>>>>> their
> >>>>>> recent modifications to the reading threads.
> >>>>>>
> >>>>>> To prevent writing threads from overwriting recent modifications
> >>>>>> before they could be read and verified, the testing framework
> >>>>>> walks
> >>>>>> the hash map keys in in a linear (or in the case of updates,
> >>>>>> circular) order. By using a hash map with a large enough
> >>>>>> capacity,
> >>>>>> readers have the time to verify the recent modifications
> >>>>>> before the
> >>>>>> writer threads come back to modify that part of the key space
> >>>>>> again.
> >>>>>>
> >>>>>> Using an adapter for the map implementation, the testing
> >>>>>> framework
> >>>>>> starts five writer threads and ten reader threads at the same
> >>>>>> time.
> >>>>>> These threads run wide open for 30 seconds, except that the
> >>>>>> readers
> >>>>>> will give up their time slice if they find nothing on the
> >>>>>> queue. The
> >>>>>> HashMaps were all sized for the needed capacity upon creation,
> >>>>>> so no
> >>>>>> resizing occurred during testing.
> >>>>>>
> >>>>>> I got some interesting results.
> >>>>>>
> >>>>>> Four implementations were tested, Java's unsynchronized HashMap
> >>>>>> implementation, Java's synchronized HashMap implementation,
> >>>>>> Java's
> >>>>>> ConcurrentHashMap implementation, and OpenJPA's ConcurrentHashMap
> >>>>>> implementation.
> >>>>>>
> >>>>>> Only Java's unsynchronized HashMap failed, as expected, under
> >>>>>> test.
> >>>>>> Under test, this implementation demonstrates its inability to
> >>>>>> handle
> >>>>>> concurrency. The other three implementations worked flawlessly
> >>>>>> under
> >>>>>> test.
> >>>>>>
> >>>>>> The java.util.concurrent.ConcurrentHashMap implementation
> >>>>>> (available
> >>>>>> with Java 5 and 6) was the fastest implementation tested.
> >>>>>>
> >>>>>> Java's synchronized wrapper for the HashMap implementation is
> >>>>>> one to
> >>>>>> two orders of magnitude slower than Java's ConcurrentHashMap
> >>>>>> implementation.
> >>>>>>
> >>>>>> OpenJPA's ConcurrentHashMap compares equally with Java's
> >>>>>> ConcurrentHashMap in find operations and is 2-4 times slower
in
> >>>>>> mutating operations.
> >>>>>>
> >>>>>> Implementation   Add   Remove   Update  Find-a  Find-r  Find-u
> >>>>>> ---------------+------+-------+--------+-------+-------+------
> >>>>>> synchronized     103     35       50      40      37     54
> >>>>>> concurrent      13.2    6.4      6.1     0.6     0.3    1.1
> >>>>>> OpenJPA         29.8   26.6     27.9     0.6     0.6    0.6
> >>>>>>
> >>>>>>
> >>>>>> Legend:
> >>>>>>
> >>>>>> synchronized:
> >>>>>> java.util.Collections.synchronizedMap(new java.util.HashMap())
> >>>>>> concurrent: java.util.concurrent.ConcurrentHashMap
> >>>>>> OpenJPA: org.apache.openjpa.lib.util.concurrent.ConcurrentHashMap
> >>>>>>
> >>>>>> Add: time for average add operation
> >>>>>> Remove: time for average remove operation
> >>>>>> Update: time for average update of new value for existing key
> >>>>>> Find-a: time to find a recent addition
> >>>>>> Find-r: time to NOT find a recent removal
> >>>>>> Find-u: time to find a recent update
> >>>>>>
> >>>>>> These times (in microseconds) are representative, but are not
the
> >>>>>> average of several runs. The tests were run on a Dell Dual Core
> >>>>>> laptop under Windows. The performance meter was pegged during
the
> >>> tests.
> >>>>>>
> >>>>>> David Ezzio
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >
> >
> > Notice:  This email message, together with any attachments, may
> > contain information  of  BEA Systems,  Inc.,  its subsidiaries
> > and  affiliated entities,  that may be confidential,  proprietary,
> > copyrighted  and/or legally privileged, and is intended solely for
> > the use of the individual or entity named in this message. If you
> > are not the intended recipient, and have received this message in
> > error, please immediately return this by email and then delete it.
>
>


-- 
Patrick Linskey
202 669 5907

Mime
View raw message