hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiroshi Ikeda (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
Date Wed, 05 Mar 2014 08:11:53 GMT

     [ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Hiroshi Ikeda updated HBASE-10656:

    Attachment: MyCounter2.java

Added an alternative counter class, using some logic from LongAdder in Java8.

The counter reuse resources when extending the internal hash table, so that it avoids complexity
of race condition caused by the extending, and it avoid to hold old unused resources and their
sum cache for performance. Instead, the counter has to create an instance per each value holder.
I think this may be overhead, but each instance can be independently placed in memory so that
this may help to reduce cache line contention. 

Like LongAdder, each value holder is surrounded by 2 of fixed size pads. Each of the pads
consists of 7 long values (64bit * 7). That means, this expects the size of cache line is
fixed to about 64bit * 16.

This counter simply uses Thread.getId() as hashcode. The reversed bit of the id is well spreaded,
but cache line contention is not avoidable before every extending the hash table if the estimate
size of cache line is small. With starting small hash size, this may come frequently in the
early stages, and anyone don't know whether the extendings will be stopped at a good point
or not.

Unlike LongAdder, when this counter creates the internal hash table (that is an array of references
to value holders), this counter fills it with value holder instances. Creating instances on
demand is too complex to keep consistency. In source code of LongAdder I referred, I didn't
find out what ensures happens-before relation between putting value holders on demand to the
internal array and referring them from the array in order to sum up the values.

Besides the above concerns, in my environment, the added counter seems to have enough performance
to (a bit faster than) high-scale-lib even without mixing read operations.

>  high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
> ----------------------------------------------------------------------------
>                 Key: HBASE-10656
>                 URL: https://issues.apache.org/jira/browse/HBASE-10656
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Hiroshi Ikeda
>            Priority: Minor
>         Attachments: MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java
> Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion)
in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard
library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific
JRE Implementation.
> To make matters worse, Counter has a bug and you may get wrong result if you mix a reading
method into your logic calling writing methods.
> In more detail, I think the bug is caused by reading an internal array field without
resolving memory caching, which is intentional the comment says, but storing the read result
into a volatile field. That field may be not changed after you can see the true values of
the array field, and also may be not changed after updating the "next" CAT instance's values
in some race condition when extending CAT instance chain.
> Anyway, it is possible that you create a new alternative class which only depends on
the standard library. I know Java8 provides its alternative, but HBase should support Java6
and Java7 for some time.

This message was sent by Atlassian JIRA

View raw message