hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject [DISCUSS] No regions on Master node in 2.0
Date Fri, 08 Apr 2016 05:11:36 GMT
I would like to start a discussion on whether Master should be carrying
regions or not. No hurry. I see this thread going on a while and what with
2.0 being a ways out yet, there is no need to rush to a decision.

First, some background.

Currently in the master branch, HMaster hosts 'system tables': e.g.
hbase:meta. HMaster is doing more than just gardening the cluster,
bootstrapping and keeping all up and serving healthy as in branch-1; in
master branch, it is actually in the write path for the most critical
system regions.

Master is this way because HMaster and HRegionServer servers have so much
in common, they should be just one binary, w/ HMaster as any other server
with the HMaster function a minor appendage runnable by any running
HRegionServer.

I like this idea, but the unification work was just never finished. What is
in master branch is a compromise. HMaster is not a RegionServer but a
sort-of RegionServer doing part serving. So we have HMaster role, a new
part-RegionServer-carrying-special-regions role and then a full-on
HRegionServer role. We need to fix this messyness. We could revert to plain
branch-1 roles or carrying the
HMaster-function-is-something-any-RegionServer-could-execute through to
completion.

More background from a time long-past with good comments by the likes of
our Francis Liu and Mighty Matteo Bertozzi are here [1], on unifying master
and meta-serving. Slightly related are old discussions on being able to
scale by splitting meta with good comments by our Elliott Clark [2].

Also for consideration, the landscape has since changed. [1] was written
before we had ProcedureV2 available to us where we could record
intermediate transition states local to the Master rather than remote as
intermediate updates to an hbase:meta over rpc running on another node.

Enough on the background.

Let me provoke discussion by making the statement that we should undo
HMaster carrying any regions ever; that the HMaster function is work enough
for a single dedicated server and that it important enough that it cannot
take a background role on a serving RegionServer (I could go back from this
position if evidence HMaster role could be backgrounded). Notions of a
Master carrying system tables only are just not on given system tables will
be too big for a single server especially when hbase:meta is split (so we
can scale). This simple distinction of HMaster and RegionServer roles is
also what our users know and have gotten used to so needs to be a good
reason to change it (We can still pursue the single binary that can do
HMaster or HRegionServer role determined at runtime).

Thanks,
St.Ack

1.
https://docs.google.com/document/d/1xC-bCzAAKO59Xo3XN-Cl6p-5CM_4DMoR-WpnkmYZgpw/edit#heading=h.j5yqy7n04bkn
2.
https://docs.google.com/document/d/1eCuqf7i2dkWHL0PxcE1HE1nLRQ_tCyXI4JsOB6TAk60/edit#heading=h.80vcerzbkj93

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message