hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Antonov <olorinb...@gmail.com>
Subject Re: [DISCUSS] No regions on Master node in 2.0
Date Wed, 16 Nov 2016 11:06:25 GMT
(side note @Yu - FYI there has been a number of fixes/optimizations to the
way we cache and invalidate caches of region locations on the client side
in 1.3, see for example HBASE-15658, HBASE-15654).

On the topic -

Hosting meta on the machine that doesn't serve user regions would help to
ensure that updates to meta have higher chance to succeed. But if that
machine isn't Master, then we'd introduce yet one more service role to the
deployment. And I'd say that different machine roles/service types required
for HBase deployment is something we already have enough of.

I think this discussion is still at the same point as it was back then - it
looks like we're essentially comparing (A) an existing feature that works
and has practical benefits (as noted above on the thread) to the (B)
different way of doing things that's not finalized / released yet (please
correct me if I'm wrong)?

And assuming B is finalized, I'm not sure that it actually fully addresses
the problems that A addresses now. That makes me inclined to think that
removing option A before we know that the actual problems it solves now are
completely addressed by other means would put us in a bad state.

-Mikhail

On Wed, Nov 16, 2016 at 2:13 AM, Yu Li <carp84@gmail.com> wrote:

> Very late to the party +1 (Smile)
>
> We also offline discussed standalone meta server here in Alibaba since
> we've observed crazily high QPS on meta caused by online machine learning
> workload, and in the discussion we also mentioned pros. and cons. of
> serving meta on HMaster. Since quite some pros. already mentioned in the
> thread, I'd like to mention one cons. here: currently we could switch
> active master (almost) freely w/o affecting online service, so we could do
> some hot-fix on master. But if we carry meta region on HMaster, the cost of
> switching master will increase a lot and the hot-switch may not be possible
> any more. Not sure whether this is an important thing for most users but
> still a point to share (Smile).
>
> And maybe another point for discussion: if not placed on HMaster, should we
> have a standalone meta server or at least provide such an option?
>
> Thanks.
>
> Best Regards,
> Yu
>
> On 16 November 2016 at 03:43, <toffer@ymail.com.invalid> wrote:
>
> > > In the absence of more information, intuition says master carries meta
> > to avoid a whole class of problems.
> > Off-hand I think the class of problems we'll eliminate are problems that
> > are well understood and being constantly dealt with and hardened to this
> > day (ie puts to a region).
> > > I think we have to evaluate whether the new pv2 master works with
> > remote meta updates and the fact that those updates can fail partially or
> > succeed without theI think failing meta updates need to be dealt with
> > either way AFAIK eventually procedure state will be stored in HDFS which
> is
> > also a distributed system.
> >
> >
> >
> >     On Saturday, November 12, 2016 9:45 AM, Andrew Purtell <
> > apurtell@apache.org> wrote:
> >
> >
> >  Thanks Stack and Enis. I concur, it's hard to say for those not intimate
> > with the new code.
> >
> > In the absence of more information, intuition says master carries meta to
> > avoid a whole class of problems.
> >
> > On Fri, Nov 11, 2016 at 3:27 PM, Enis Söztutar <enis@apache.org> wrote:
> >
> > > Thanks Stack for reviving this.
> > >
> > > How to move forward here? The Pv2 master is almost done. An ITBLL
> bakeoff
> > > > of new Pv2 based assign vs a Master that exclusively hosts
> hbase:meta?
> > > >
> > > >
> > > I think we have to evaluate whether the new pv2 master works with
> remote
> > > meta
> > > updates and the fact that those updates can fail partially or succeed
> > > without the
> > > client getting the reply, etc. Sorry it has been some time I've looked
> at
> > > the design.
> > > Actually what would be very good is to have a design overview / write
> up
> > of
> > > the pv2
> > > in its current / final form so that we can evaluate. Last time I've
> > looked
> > > there was no
> > > detailed design doc at all.
> > >
> > >
> > > > St.Ack
> > > >
> > > >
> > > >
> > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >   - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
> >
> >
>



-- 
Thanks,
Michael Antonov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message