hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Antonov <olorinb...@gmail.com>
Subject Re: [DISCUSS] Dynamic switches in master
Date Thu, 04 Feb 2016 06:31:15 GMT
Looked through the comments and glanced at the patch.

I generally think that the patch addressing specific existing problem
shouldn't be blocked on generic big feature like dynamic configuration
framework, so would be good to get something committable sooner rather then
latter.

It seems the only point of controversy is whether we should have generic
setSwitch kind of call and kind of open up some API for that sort of
dynamic configuration, or rather specific setBalancer(),
forbidSplitsAndMerges().

So short term, why not to get a patch without generic kind of switches
(should be pretty easy change?), seems like people would agree on it,
commit it to fix existing issue, and then move on to generalize dynamic
configuration?

I tend to agree that creating api to store generic switches in ZK while not
abandoning idea of generic dynamic configuration framework looks like
slippery slope. Right now we have well-known N things kept in ZK (I admit I
have contributed to that number), but I wouldn't have unknown X things
there.

(Speaking of which.. I think most people would like to be able to hotswap
most of configuration properties without having to roll all the servers,
and do that from rpc/shell/some other interface, not just reload conf file.
In that case we would need to decide first on the persistent store for it
and consistency around it.

Ideally we may want to have something committable and rollback-able with
versions? To cover cases like: I changed the parameter, say rpc timeout,
from command line, now I expect it to persist and survive over the next
cluster restart. But what if I don't like the change I made, I would want
to easily discard the change and say "whatever was in configuration was
good, let's go back to it"?)

-Mikhail

On Wed, Feb 3, 2016 at 9:50 PM, 张铎 <palomino219@gmail.com> wrote:

> My opinion, use conf file as default, and use code to override the config
> in conf file.
> And we need to persist the overridden value somewhere, not zk I'd say. I
> agree with Matteo that we should reduce the dependency of zk. Maybe we
> could introduce an hbase:config table?
>
> And for the dynamic configuration framework, it is difficult to treat all
> configurations in the same way I think. I prefer changing all
> configurations with a shell command(the actual implementation should an RPC
> call to master maybe), but if we have regionservers with different
> hardwares in cluster, they may have different configurations so modify conf
> file directly on the machine is a more proper way? I have no idea...
>
> Thanks.
>
> 2016-02-04 11:40 GMT+08:00 Enis Söztutar <enis@hortonworks.com>:
>
> > This came up in the review for HBASE-15128, but we thought that we should
> > surface this discussion in dev@.
> >
> > The idea for HBASE-15128 is that, we should have a dynamic switch in
> master
> > so that the operator (or HBCK) can disable all region splits or merges.
> >
> > We currently have such dynamic switches for balancer and normalizer which
> > keep their state in zk. We also have a switch for catalog janitor which
> > keeps its state only in memory (not sure why). For the new switches Heng
> > Chen implemented a "set_switch" RPC to master, a switch type, a zk node.
> > The idea is to move balancer, catalog janitor and normalizer switches to
> > this after this patch.
> >
> > However, Matteo raised a point about this framework being an alternative
> > way of doing dynamic configuration. The problem is that we do not have a
> > dynamic conf framework that we can piggyback on, and since balancer and
> > other switches already work this way, it does not make sense to block the
> > issue. What do you guys think? If we want to implement dynamic conf, how
> > would we handle things like enabling / disabling balancer from conf and
> or
> > from code? Any good ideas?
> >
> > Enis
> >
>



-- 
Thanks,
Michael Antonov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message