openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Linskey" <plins...@gmail.com>
Subject Re: Comparison Metrics for OpenJPA's ConcurrentHashMap
Date Sat, 02 Jun 2007 04:07:04 GMT
Hi,

How many cores / CPUs were you using in these tests?

-Patrick

On 6/1/07, David Ezzio (asmtp) <dezzio@bea.com> wrote:
> Hi Marc,
>
> I still was not able to get HighScaleLib to work. It produces a
> SecurityException when attempting to get the Unsafe object. I decided to
> avoid changing the relevant security policy.
>
> On the other hand, I did test Emory University's backport of the
> java.util.concurrent package. This provides to Java 1.4 a
> ConcurrentHashMap implementation that is compatible with the one found
> in Java 5 and 6.
>
> I also realized that I could get another metric from my numbers, the
> percentage of time the threads were suspended during the metered
> operations. Then I reran the tests for Emory's backport using fewer
> threads. In the classic pattern of an overloaded CPU, the higher thread
> count both lowers throughput and increases response time.  (Throughput
> is yet another number I haven't extracted from the data, although it is
> obvious when running the tests.)
>
> All of the previous tests were run with 5 writing and 10 reading
> threads. Only backport was additionally tested with 2 writing and 4
> reading threads. The suspended percentage is approximate since the
> adding and updating tests have slightly different numbers.
>
> Implementation  Add   Remove  Update  Find-a  Find-r  Find-u  Suspended
> --------------+------+-------+-------+-------+-------+-------+---------
> synchronized     103     35      50      40      37     54
> concurrent      13.2    6.4     6.1     0.6     0.3    1.1
> OpenJPA         29.8   26.6    27.9     0.6     0.6    0.6
> Backport (5/10) 43.2   36.8    40.4     0.3     0.2    0.5       62%
> Backport (2/4)   6.1    3.1     3.3     0.3     0.2    0.3        4%
>
> David
>
>
> David Ezzio (asmtp) wrote:
> > Hi Marc,
> >
> > I did plug it in, but it failed straightaway on a security issue.  I
> > should probably read its documentation. :)  I'll try it again along with
> > the backport lib done by Emory U.
> >
> > David
> >
> >
> > Marc Prud'hommeaux wrote:
> >> David-
> >>
> >> That is very interesting.
> >>
> >> Did you also take a look at the one at
> >> http://sourceforge.net/projects/high-scale-lib ? They say its
> >> performance only shines for high thread/cpu counts, but it might be
> >> interesting to see where its numbers lie in the range.
> >>
> >>
> >>
> >> On May 29, 2007, at 11:01 AM, David Ezzio (asmtp) wrote:
> >>
> >>> Recently, I did some testing of Map implementations under concurrency.
> >>>
> >>> My primary purpose was to verify the reliability of OpenJPA's
> >>> ConcurrentHashMap implementation. As I got into it, I saw the
> >>> opportunity to get some performance metrics out of the test.
> >>>
> >>> The biggest part of my task was coming up with a reliable and useful
> >>> testing framework. I design it with the following two factors in
> >>> mind: First, I wanted to test the edge conditions where an entry had
> >>> just been added or removed or where a key's value had just been
> >>> updated. The idea is that a number of threads add, remove, and update
> >>> entries, while other threads check to see if these recent
> >>> modifications are visible (or in the case of removals, not visible).
> >>> Second, I wanted the testing framework itself to be free of
> >>> synchronization. If the testing framework used synchronization then
> >>> it would tend to serialize the readers and writers and thereby mask
> >>> concurrency issues in the map implementation under test.
> >>>
> >>> The testing framework uses a non-synchronizing, non-blocking FIFO
> >>> queue as the mechanism for the writing threads to communicate their
> >>> recent modifications to the reading threads.
> >>>
> >>> To prevent writing threads from overwriting recent modifications
> >>> before they could be read and verified, the testing framework walks
> >>> the hash map keys in in a linear (or in the case of updates,
> >>> circular) order. By using a hash map with a large enough capacity,
> >>> readers have the time to verify the recent modifications before the
> >>> writer threads come back to modify that part of the key space again.
> >>>
> >>> Using an adapter for the map implementation, the testing framework
> >>> starts five writer threads and ten reader threads at the same time.
> >>> These threads run wide open for 30 seconds, except that the readers
> >>> will give up their time slice if they find nothing on the queue. The
> >>> HashMaps were all sized for the needed capacity upon creation, so no
> >>> resizing occurred during testing.
> >>>
> >>> I got some interesting results.
> >>>
> >>> Four implementations were tested, Java's unsynchronized HashMap
> >>> implementation, Java's synchronized HashMap implementation, Java's
> >>> ConcurrentHashMap implementation, and OpenJPA's ConcurrentHashMap
> >>> implementation.
> >>>
> >>> Only Java's unsynchronized HashMap failed, as expected, under test.
> >>> Under test, this implementation demonstrates its inability to handle
> >>> concurrency. The other three implementations worked flawlessly under
> >>> test.
> >>>
> >>> The java.util.concurrent.ConcurrentHashMap implementation (available
> >>> with Java 5 and 6) was the fastest implementation tested.
> >>>
> >>> Java's synchronized wrapper for the HashMap implementation is one to
> >>> two orders of magnitude slower than Java's ConcurrentHashMap
> >>> implementation.
> >>>
> >>> OpenJPA's ConcurrentHashMap compares equally with Java's
> >>> ConcurrentHashMap in find operations and is 2-4 times slower in
> >>> mutating operations.
> >>>
> >>> Implementation   Add   Remove   Update  Find-a  Find-r  Find-u
> >>> ---------------+------+-------+--------+-------+-------+------
> >>> synchronized     103     35       50      40      37     54
> >>> concurrent      13.2    6.4      6.1     0.6     0.3    1.1
> >>> OpenJPA         29.8   26.6     27.9     0.6     0.6    0.6
> >>>
> >>>
> >>> Legend:
> >>>
> >>> synchronized:
> >>> java.util.Collections.synchronizedMap(new java.util.HashMap())
> >>> concurrent: java.util.concurrent.ConcurrentHashMap
> >>> OpenJPA: org.apache.openjpa.lib.util.concurrent.ConcurrentHashMap
> >>>
> >>> Add: time for average add operation
> >>> Remove: time for average remove operation
> >>> Update: time for average update of new value for existing key
> >>> Find-a: time to find a recent addition
> >>> Find-r: time to NOT find a recent removal
> >>> Find-u: time to find a recent update
> >>>
> >>> These times (in microseconds) are representative, but are not the
> >>> average of several runs. The tests were run on a Dell Dual Core
> >>> laptop under Windows. The performance meter was pegged during the tests.
> >>>
> >>> David Ezzio
> >>>
> >>>
> >>
> >>
> >
> >
>
>


-- 
Patrick Linskey
202 669 5907

Mime
View raw message