hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14708) Use copy on write TreeMap for region location cache
Date Thu, 29 Oct 2015 05:36:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979852#comment-14979852

Elliott Clark commented on HBASE-14708:

bq.CopyOnWriteTreeMap is not required to implement Map nor ConcurrentNavigableMap
Quite a few of the methods of those interfaces are used in the meta cache. So it felt better
to just use the interfaces and create something that could be more useful other places. There
are already a few places that I want to look into using this as CSM is used all over the place
in hbase.

bq.Using AtomicReference instead of synchronized for the internal map might be better
So I benchmarked that. They end up looking pretty close to the same. Reading/Writing to either
an atomic reference or a volatile field both require a memory barrier. Synchronized with a
volatile came out slightly ahead for the read. Thought I am not sure why. Either way it should
be negligible.

If I went with the atomic reference I would have gone with the compare and swap. It would
have bloated the size of each method. So without a clear benefit I went for the easiest to

> Use copy on write TreeMap for region location cache
> ---------------------------------------------------
>                 Key: HBASE-14708
>                 URL: https://issues.apache.org/jira/browse/HBASE-14708
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>    Affects Versions: 1.1.2
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Critical
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>         Attachments: HBASE-14708-v2.patch, HBASE-14708-v3.patch, HBASE-14708-v4.patch,
HBASE-14708-v5.patch, HBASE-14708-v6.patch, HBASE-14708-v7.patch, HBASE-14708.patch, location_cache_times.pdf,
> Internally a co-worker profiled their application that was talking to HBase. > 60%
of the time was spent in locating a region. This was while the cluster was stable and no regions
were moving.
> To figure out if there was a faster way to cache region location I wrote up a benchmark
here: https://github.com/elliottneilclark/benchmark-hbase-cache
> This tries to simulate a heavy load on the location cache. 
> * 24 different threads.
> * 2 Deleting location data
> * 2 Adding location data
> * Using floor to get the result.
> To repeat my work just run ./run.sh and it should produce a result.csv
> Results:
> ConcurrentSkiplistMap is a good middle ground. It's got equal speed for reading and writing.
> However most operations will not need to remove or add a region location. There will
be potentially several orders of magnitude more reads for cached locations than there will
be on clearing the cache.
> So I propose a copy on write tree map.

This message was sent by Atlassian JIRA

View raw message