hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: [DISCUSS] No regions on Master node in 2.0
Date Wed, 16 Nov 2016 13:29:08 GMT
Gary has a JIRA HBASE-16025 which would reduce the load on server hosting hbase:meta. 

FYI

> On Nov 16, 2016, at 2:13 AM, Yu Li <carp84@gmail.com> wrote:
> 
> Very late to the party +1 (Smile)
> 
> We also offline discussed standalone meta server here in Alibaba since
> we've observed crazily high QPS on meta caused by online machine learning
> workload, and in the discussion we also mentioned pros. and cons. of
> serving meta on HMaster. Since quite some pros. already mentioned in the
> thread, I'd like to mention one cons. here: currently we could switch
> active master (almost) freely w/o affecting online service, so we could do
> some hot-fix on master. But if we carry meta region on HMaster, the cost of
> switching master will increase a lot and the hot-switch may not be possible
> any more. Not sure whether this is an important thing for most users but
> still a point to share (Smile).
> 
> And maybe another point for discussion: if not placed on HMaster, should we
> have a standalone meta server or at least provide such an option?
> 
> Thanks.
> 
> Best Regards,
> Yu
> 
> On 16 November 2016 at 03:43, <toffer@ymail.com.invalid> wrote:
> 
>>> In the absence of more information, intuition says master carries meta
>> to avoid a whole class of problems.
>> Off-hand I think the class of problems we'll eliminate are problems that
>> are well understood and being constantly dealt with and hardened to this
>> day (ie puts to a region).
>>> I think we have to evaluate whether the new pv2 master works with
>> remote meta updates and the fact that those updates can fail partially or
>> succeed without theI think failing meta updates need to be dealt with
>> either way AFAIK eventually procedure state will be stored in HDFS which is
>> also a distributed system.
>> 
>> 
>> 
>>    On Saturday, November 12, 2016 9:45 AM, Andrew Purtell <
>> apurtell@apache.org> wrote:
>> 
>> 
>> Thanks Stack and Enis. I concur, it's hard to say for those not intimate
>> with the new code.
>> 
>> In the absence of more information, intuition says master carries meta to
>> avoid a whole class of problems.
>> 
>>> On Fri, Nov 11, 2016 at 3:27 PM, Enis Söztutar <enis@apache.org> wrote:
>>> 
>>> Thanks Stack for reviving this.
>>> 
>>> How to move forward here? The Pv2 master is almost done. An ITBLL bakeoff
>>>> of new Pv2 based assign vs a Master that exclusively hosts hbase:meta?
>>> I think we have to evaluate whether the new pv2 master works with remote
>>> meta
>>> updates and the fact that those updates can fail partially or succeed
>>> without the
>>> client getting the reply, etc. Sorry it has been some time I've looked at
>>> the design.
>>> Actually what would be very good is to have a design overview / write up
>> of
>>> the pv2
>>> in its current / final form so that we can evaluate. Last time I've
>> looked
>>> there was no
>>> detailed design doc at all.
>>> 
>>> 
>>>> St.Ack
>>>> 
>>>> 
>>>> 
>>>>> 
>> 
>> 
>> 
>> --
>> Best regards,
>> 
>>  - Andy
>> 
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>> 
>> 
>> 

Mime
View raw message