hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14460) [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed Increments, CheckAndPuts, batch operations
Date Sat, 12 Dec 2015 23:31:46 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-14460:
--------------------------
    Attachment: testincrement.094.patch
                testincrement.098.patch
                testincrement.master.patch
                flamegraph-9466.094.singlecell.svg
                flamegraph-13120.svg.master.singlecell.svg
                flamegraph-28066.098.singlecell.svg

There are two ways in which master is slower than 0.94 increments. There is the case where
threads are contending to update a single Cell and then there is the case described at the
head of this issue where the mvcc coordination is acting like a region-wide lock though all
threads incrementing may not be contending on a Cell.

Here are some rough measurements of the first case. See attached test. It has 100 threads
doing 10k increments of a single Cell up against a Region Instance.

{code}
0.94 ~84 seconds
0.98 ~140 seconds
master ~180 seconds
{code}

0.98 is almost 2x slower than 0.94 (though the code path profile is pretty close if you look
at the accompanying flame graphs) and master is slower again, more than 2x slower.

As is, reports from the field have it that even 0.98 increments are too slow as is (being
2x slower, if many, can back up all handlers so no other work can get in). Hence the above
exercise. It seem that indeed even without mvcc unification, increments have gotten slower.

Let me go measure the case where mvcc is getting in the way next.

> [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed Increments,
CheckAndPuts, batch operations
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14460
>                 URL: https://issues.apache.org/jira/browse/HBASE-14460
>             Project: HBase
>          Issue Type: Bug
>          Components: Performance
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>         Attachments: 14460.txt, flamegraph-13120.svg.master.singlecell.svg, flamegraph-28066.098.singlecell.svg,
flamegraph-9466.094.singlecell.svg, region_lock.png, testincrement.094.patch, testincrement.098.patch,
testincrement.master.patch
>
>
> As reported by 鈴木俊裕 up on the mailing list -- see "Performance degradation between
CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" -- our unification of sequenceid and MVCC
slows Increments (and other ops) as the mvcc needs to 'catch up' to our current point before
we can read the last Increment value that we need to update.
> We can say that our Increment is just done wrong, we should just be writing Increments
and summing on read, but checkAndPut as well as batching operations have the same issue. Fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message