hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12233) Bring back root table
Date Sat, 11 Oct 2014 01:37:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167927#comment-14167927
] 

Elliott Clark commented on HBASE-12233:
---------------------------------------

So right now I don't see the need for split meta. We found a 5x win on meta versions. This
alone means that we can handle lots more regions per cluster with out breaking a sweat. And
there's lots more that can be done without hurting the default case.

* We haven't tried meta region replicas.
** This is where we will see the biggest wins. Right now meta gets 2000x more reads than writes
on real production clusters.
** We already have the region replica stuff being worked on
** All reads to meta can already be stale so timeline consistent is good enough for >99%
of reads that are currently going to meta.
* We haven't tried allowing a shared cache infront of meta.
** It would be easy to put memcache (or redis if you already have it) in the MetaTableAccessor
and that would allow a very fast shared cache that gets warmed by all clients.
* We haven't tried a more compact meta table representation
* We haven't tried picking smaller split keys.
** Right now we just pick a mid point. It would be possible to do something like our index
key generation to drastically reduce the size of region names.
* We haven't tried stripe compactions on meta.
* We haven't gotten block encoding working for meta.

Each of those things are largish wins that haven't been tried, and each of those things helps
every user. While split meta helps maybe 0.1% of HBase's users.

Until the less invasive more positive options  are tried and shown to not be enough, it just
feels like everyone is trying to split meta without a concrete need.

Splitting meta doesn't make us able to scale to the sky. The NN is still a bottle neck on
cluster size. NameNodes fall over way before we will need a split meta.  Even if someone gets
NN to scale to infinity, the network becomes a bottleneck. HDFS clusters can only get so large
before they need to be federated (and HBase doesn't tolerate federation). So making every
user and every cluster feel the pain of increased complexity and mttr so that we the HBase
developers can perform a thought experiment is just not something I see the benefit in right
now.

I am for sure against bringing root back in for branch-1 ( -1 ). It's just way too late in
the stabilization period to not set us way back.

> Bring back root table
> ---------------------
>
>                 Key: HBASE-12233
>                 URL: https://issues.apache.org/jira/browse/HBASE-12233
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Virag Kothari
>            Assignee: Virag Kothari
>         Attachments: HBASE-12233.patch
>
>
> First step towards splitting meta.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message