hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Shin <kevin.s...@thinkbiganalytics.com>
Subject Improving Coprocessor postSplit/postOpen synchronization
Date Mon, 27 Aug 2012 20:29:15 GMT
Hi everyone,

A colleague and I were working with HBase coprocessors for secondary
indexes and ran into an interesting problem regarding splits
and synchronizing the corresponding parent/daughter regions.

The goal with splits is to create two new daughter regions with the
corresponding splits of the secondary indexes and lock these regions such
that Puts/Deletes that occur while postSplit is in progress will be queued
up so we don't run into consistency issues. IE, if a delete gets called
before a daughter region receives the split index, that delete would
essentially have been ignored, so we would want to wait until postSplit is
finished before running any new Puts/Deletes on the split regions.

As of right now, the HBase coprocessors do not easily support a way to
achieve this level of consistency in that there is no way to distinguish a
region being opened from a split or a regular open. If we could
distinguish, we could open up the correct index from the start and stall
until postSplit is finished in the background in the event of a split. I
would thus like to propose a way to "lock" the daughter regions when
postSplit is called. That is, when we open a daughter region from a split,
we can pass in the parent region name alongside it (or Null if there is no
parent) to distinguish a region being opened from a split or open. I am
thinking about submitting a patch into JIRA but would greatly appreciate
any thoughts or suggestions for another solution to the problem or perhaps
a better patch. I am using HBase 0.92 for development at this moment.

Best,
Kevin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message