hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15158) Change order in which we do write pipeline operations; do all under row locks!
Date Thu, 28 Jan 2016 22:37:40 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-15158:
    Attachment: 15158v2.patch

Took me a while. All tests that failed now pass locally.

TestHRegionReplayEvents was 'fixed' by pushing through more edits; localfs buffer was not
flushing out all edits for the test (This item may come back to bite me.. can't figure why
this patch brings this on... and it is this single test only... we'll see). Other items were
fixed by careful compare of patch and old code... I'd not restored the replay code 100%.

I can break this patch up. Let me do that.

> Change order in which we do write pipeline operations; do all under row locks!
> ------------------------------------------------------------------------------
>                 Key: HBASE-15158
>                 URL: https://issues.apache.org/jira/browse/HBASE-15158
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: stack
>            Assignee: stack
>             Fix For: 2.0.0
>         Attachments: 15158.patch, 15158v2.patch
> Change how we do our write pipeline. I want to do all write pipeline ops under row lock
so I lean on this fact fixing performance regression in check-and-set type operations like
increment, append, and checkAnd* (see sibling issue HBASE-15082).
> To be specific, we write like this now:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # add to memstore
> # let go of rowlock
> # sync WAL
> # in case of error: rollback memstore
> {code}
> Instead, write like this:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # sync WAL
> # add to memstore
> # let go of rowlock
> ... no need to do rollback.
> {code}
> The old ordering was put in place because it got better performance in a time when WAL
was different and before row locks were read/write (HBASE-12751).
> Testing in branch-1 shows that a reordering and skipping mvcc waits gets us back to the
performance we had before we unified mvcc and sequenceid (HBASE-8763). Tests in HBASE-15046
show that at the macro level using our usual perf tools, reordering pipeline seems to cause
no slowdown (see HBASE-15046). A rough compare of increments with reordered write pipeline
seems to have us getting back a bunch of our performance (see tail of https://issues.apache.org/jira/browse/HBASE-15082?focusedCommentId=15111703&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15111703
and subsequent comment).

This message was sent by Atlassian JIRA

View raw message