hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Pull instant schema updating out?
Date Tue, 03 Apr 2012 00:45:52 GMT
Agree about pulling out something that is broken.

However, it should at least be possible to update 
via the shell parameters such as split threshold with instantaneous 
effect and no region offlining. I've encountered situations where it is really necessary to
be able to alter the attributes of existing tables without the "disruptions in the force"
of a rolling cluster configuration update. Things like adjusting the split threshold of a
table growing a lot larger than anticipated (while obviously given the write rate region availability
is important). 

I don't have an opinion about addition or removal of column families, but online CF addition
could be useful when evolving application design yet maintaining high availability.

Best regards,

    - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)

> From: Jean-Daniel Cryans <jdcryans@apache.org>
>To: dev@hbase.apache.org 
>Sent: Monday, April 2, 2012 4:56 PM
>Subject: Pull instant schema updating out?
>Hi nifty devs,
>After encountering HBASE-5702, I started playing with instant schema
>updating (HBASE-4213) a bit more and I must say that it's a bit rough
>which makes me wonder... should we pull that code out?
>We're in this "interesting" situation in 0.94 where we have two
>different ways to alter tables without disabling them and I don't
>trust either. I'm pretty sure most of the devs don't even know which
>one takes precedence over the other when both are enabled without
>looking at the code. Well, right now hbase.online.schema.update.enable
>needs to be enabled in order to have
>hbase.instant.schema.alter.enabled working. If only the former is
>enabled the master handles the alter, else if both are enabled then
>it's going to be done via ZK although the master still keeps track of
>So the differences between both IIUC:
>- "Online schema update" is a rolling close/open of all the regions
>so that they pick up the new HTD. It's handled by the master and has
>been in since 0.92. I've used it quite a bit when running other tests
>and it's ok as long as regions are not splitting and RS are not
>shutting down. We also enabled it on our clusters here since our
>regions don't tend to move that much.
>- "Instant schema alter" is instant in the sense that all the regions
>are asked to close from the get-go but effectively the region servers
>can only close one region at a time. The state of the alter is kept in
>ZK and the master has a bunch of watches and logs the progress. It's
>new since 0.94 and I'm not sure if anyone is using it. I've tested it
>a bit and at the moment I can say the the MonitedTasks handling needs
>to be redone, it logs way too much information in the log, but the few
>alters I ran worked... It's just a bit hard to know when they're done.
>FWIW we could pull either or both out, but instant schema alter hasn't
>been in a released version yet so it's unlikely it'll bother someone
>while the other is already in use (like here).
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message