hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16873) WAL: SequenceId assign with less friction
Date Sun, 22 Jan 2017 09:11:26 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833387#comment-15833387

Yu Li commented on HBASE-16873:

Converting this JIRA from subtask of HBASE-16698 to an improvement, should be a follow up
of HBASE-16698 instead of part of it. Just let me know if you have different opinion boss
[~stack], thanks.

> WAL: SequenceId assign with less friction
> -----------------------------------------
>                 Key: HBASE-16873
>                 URL: https://issues.apache.org/jira/browse/HBASE-16873
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance, wal
>            Reporter: stack
> This is an issue to improve our current sequence id assignment. It has become complex
with lots of friction.
> In the old days, a simple notion that the single consumer thread pulling from the ringbuffer
should assign all sequenceids seemed to make sense. It probably had provenance in the old
days when there was a single sequenceid for a regionserver but seemed like a fine choice even
after the move to region-scoped sequenceids -- rather than regionserver scopce -- and then
beyond that, when region-scoped sequenceids were unified with mvcc. The rationale ran, a single
thread appending to the WAL can run without locks and this single thread being the arbiter
of order, seemed like the natural owner of the sequenceid increment.
> Along comes large-scale production deploy, HBASE-16698. It highlights an oversight in
the above reasoning; i.e. that the single RB consumer thread must pass a synchronize block
per region to do the sequence id update and the spread between the call to append and actual
assign of the sequence id on other side of the RB is forcing a severe serialization when there
is opportunity for parallellism.
> This issue is about taking this finding and doing better than the expedient fix done
on HBASE-16698. Can we do without the lock on the region getting the sequenceid as we call
append? Can we exploit the fact that the ringbuffer txid is always incrementing as is the
region mvcc/sequenceid? Can we use this fact to do region sequenceid w/o taking a lock?

This message was sent by Atlassian JIRA

View raw message