hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15158) Change order in which we do write pipeline operations; do all under row locks!
Date Fri, 22 Jan 2016 06:57:39 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-15158:
    Attachment: 15158.patch

This patch is a carry-over from HBASE-15082 with some small changes to address test failures
reported by hadoopqa. I don't have all issues squashed and I think I can break this patch
up into smaller pieces still, but I want to get an overnight hadoopqa run in... so posting
what I have.

> Change order in which we do write pipeline operations; do all under row locks!
> ------------------------------------------------------------------------------
>                 Key: HBASE-15158
>                 URL: https://issues.apache.org/jira/browse/HBASE-15158
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: stack
>            Assignee: stack
>             Fix For: 2.0.0
>         Attachments: 15158.patch
> Change how we do our write pipeline. I want to do all write pipeline ops under row lock
so I lean on this fact fixing performance regression in check-and-set type operations like
increment, append, and checkAnd* (see sibling issue HBASE-15082).
> To be specific, we write like this now:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # add to memstore
> # let go of rowlock
> # sync WAL
> # in case of error: rollback memstore
> {code}
> Instead, write like this:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # sync WAL
> # add to memstore
> # let go of rowlock
> ... no need to do rollback.
> {code}
> The old ordering was put in place because it got better performance in a time when WAL
was different and before row locks were read/write (HBASE-12751).
> Testing in branch-1 shows that a reordering and skipping mvcc waits gets us back to the
performance we had before we unified mvcc and sequenceid (HBASE-8763). Tests in HBASE-15046
show that at the macro level using our usual perf tools, reordering pipeline seems to cause
no slowdown (see HBASE-15046). A rough compare of increments with reordered write pipeline
seems to have us getting back a bunch of our performance (see tail of https://issues.apache.org/jira/browse/HBASE-15082?focusedCommentId=15111703&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15111703
and subsequent comment).

This message was sent by Atlassian JIRA

View raw message