hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@yahoo.com>
Subject Re: HBASE-4120 Was: trip report for Hadoop In China
Date Tue, 06 Dec 2011 21:55:24 GMT
I need to review this code. 

That said, why couldn't this be handled by a postOpen up all?

Best regards,

    - Andy


On Dec 7, 2011, at 5:24 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Modified subject to match future discussion.
> 
> Andy:
> In the latest patch for HBASE-4120, HRegionServer.addToOnlineRegions() is
> used to refresh region priority.
> Do you think RegionCoprocessorHost should be enhanced with more hooks, such
> as preAddToOnlineRegions() ?
> 
> Thanks
> 
> On Tue, Dec 6, 2011 at 12:50 PM, Andrew Purtell <apurtell@yahoo.com> wrote:
> 
>> Also. I was going to start separate threads but probably there is not that
>> much to discuss now or off-JIRA:
>> 
>> - I spoke with Jia Liu about HBASE-4120. After 0.92 is out I intend to
>> champion / mentor / co-develop 4120 and the follow on table allocation work
>> and target 0.94 for it. I think the RPC QoS aspect is not too
>> controversial. The allocation/reservation aspects I'd like to aim for a
>> coprocessor or at least master plugin based integration so they won't
>> impact stability for users who don't enable it. Unlike RPC QoS I suspect
>> the changes needed to core can be minimized to coprocessor framework
>> additions. Follow up in new JIRAs soon.
>> 
>> - I also met with the author of "CCIndex". He is still interested in
>> pursuing an upstreaming of this but may not have a lot of free time. The
>> current implementation powers the data cube product at Taobao with half a
>> million users IIRC, but the implementation is an invasive hack of the 0.20
>> core code. I think the vast majority of it is an application layer concern
>> that can be moved to a coprocessor implementation.  The coprocessor
>> framework will need some modification as well, specifically hooks for
>> handling IOEs taken on HFile reads.   Anyway I'm going to play around with
>> this and follow up with a JIRA.
>> 
>> Best regards,
>> 
>>   - Andy
>> 
>> 
>> On Dec 7, 2011, at 4:00 AM, Andrew Purtell <apurtell@apache.org> wrote:
>> 
>>> I also attended day 1 of Hadoop in China and would add these notes:
>>> 
>>> - Taobao has their own HA NameNode and is interested in contributing
>> their changes upstream. I spoke with a platform dev lead. I told him do so
>> now while HA is under active development rather than wait and miss an
>> opportunity to participate/contribute. I need to follow up with him with a
>> list of active HA jiras.
>>> 
>>> - Mingjie Lai presented on HBase and ZooKeeper security. I'll ask him to
>> upload his slides somewhere.
>>> 
>>> 
>>> Best regards,
>>> 
>>> 
>>>       - Andy
>>> 
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>> 
>>> 
>>> ----- Original Message -----
>>>> From: Todd Lipcon <todd@cloudera.com>
>>>> To: dev@hbase.apache.org
>>>> Cc:
>>>> Sent: Wednesday, December 7, 2011 2:59 AM
>>>> Subject: Re: trip report for Hadoop In China
>>>> 
>>>> Ah, I clicked on "hadooper.cn" directly instead of the earlier link to
>>>> the agenda. Oops :)
>>>> 
>>>> Maybe you can put them in a public S3 bucket?
>>>> 
>>>> -Todd
>>>> 
>>>> On Tue, Dec 6, 2011 at 10:56 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>> From http://hic2011.hadooper.cn/dct/page/1 (the first two lines):
>>>>> Slides would be posted after getting permission from presenters
>>>>> 
>>>>> I can share my slides. My attempt at uploading ppt to slideshare
>> failed and
>>>>> I don't have PRO account.
>>>>> 
>>>>> I think apache.org would filter attachment.
>>>>> 
>>>>> Suggestion on where I can put my slides would be appreciated.
>>>>> 
>>>>> On Tue, Dec 6, 2011 at 10:39 AM, Todd Lipcon <todd@cloudera.com>
>>>> wrote:
>>>>> 
>>>>>> Hey Ted,
>>>>>> 
>>>>>> Thanks for the trip report. Interesting stuff.
>>>>>> 
>>>>>> For those of us who can't read mandarin, can you link directly to
>>>> any
>>>>>> english slide decks? The hadooper.cn link seems to go to an empty
>> wiki
>>>>>> template page or something, if I understand Google Translate correctly
>>>>>> :)
>>>>>> 
>>>>>> -Todd
>>>>>> 
>>>>>> On Tue, Dec 6, 2011 at 10:36 AM, Ted Yu <yuzhihong@gmail.com>
>>>> wrote:
>>>>>>> Hi,
>>>>>>> There were around 1000 attendees for the first day of HiC.
>>>>>>> The enthusiasm for cloud computing, hadoop in particular, was
>>>>>> overwhelming.
>>>>>>> 
>>>>>>> Doug Cutting's keynote was well received by attendees.
>>>>>>> **Grzegorz's presentation about latest development @ Google,
>>>> Pregel in
>>>>>>> particular, was impressive.
>>>>>>> Xiaodong Zhang's keynote about DOT, an analytical model, was
>>>> interesting.
>>>>>>> 
>>>>>>> I listened to several presentations. Here're a few:
>>>>>>> 
>>>>>>> Zheng Shao talked about Puma3 architecture which his group
>>>> developed at
>>>>>>> Facebook
>>>>>>> Jerry and Liyin talked about how Facebook messaging uses Hbase.
>>>> Jerry
>>>>>>> covered some best practices in using Hbase. Liyin talked about
>>>> some
>>>>>>> features developed by their team, including Hfile V2, new bloom
>>>> filters,
>>>>>> etc
>>>>>>> Jinglong Liu from Baidu talked about how his team implemented
>>>> compression
>>>>>>> in hdfs (this is not RAID).
>>>>>>> 
>>>>>>> I met with developers / managers from Taobao, Huawei, Renren
>>>> (Chinese
>>>>>>> version of Facebook), etc and shared experience using HBase.
>>>>>>> 
>>>>>>> You can find the complete listing of talks here:
>>>>>>> http://hic2011.hadooper.cn/dct/page/65580
>>>>>>> 
>>>>>>> On my way back, I met Doug Cutting at the airport.
>>>>>>> He endorsed the idea of promoting committers from China so that
>>>> hadoop
>>>>>>> community would have organic growth.
>>>>>>> 
>>>>>>> I didn't take detailed notes. So please find the slides from
>>>> hadooper.cn
>>>>>>> 
>>>>>>> Thanks
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Todd Lipcon
>>>>>> Software Engineer, Cloudera
>>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>> 
>> 

Mime
View raw message