hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13643) Follow Google to get more 9's
Date Sat, 18 Jul 2015 10:46:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632368#comment-14632368

Lars Hofhansl commented on HBASE-13643:

So what's to do here?

* The memstore is already not flushed when it is empty (there's a little bit of preprocessing
going on, but that looks pretty lightweight)
* How precise is the last seqid we keep around? Specifically can we instantly tell when onlining
a region that there are guaranteed no logs to be replayed even before we split the logs?
* We could allocate mslab lazily perhaps. An unused memstore would then not consume any heap
until used.

> Follow Google to get more 9's
> -----------------------------
>                 Key: HBASE-13643
>                 URL: https://issues.apache.org/jira/browse/HBASE-13643
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Elliott Clark
> Ideas taken shamelessly from Google's HBasecon talk (http://hbasecon.com/agenda/):
> On failover all regions are unavailable for reads (and sometime writes) until after all
write ahead logs have been recovered. To combat that the last flushed seqid is kept around.
> Google took this one step farther and set some regions (Tablets in BigTable) as read
only. Setting a region as read only means there's no memstore. No need to flush before move,
split, or merge.
> In addition to the wins that Google got, HBase would also be able to shed some memory
pressure. Right now every region gets a memstore and with that memstore comes a mslab. Read
only regions would not need these added object. This should allow a regionserver to host lots
of cold regions without too much memory pressure.

This message was sent by Atlassian JIRA

View raw message