hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@yahoo.com>
Subject Re: HBASE-4120 Was: trip report for Hadoop In China
Date Tue, 06 Dec 2011 21:56:48 GMT
Upcall 

iPhone automiscorrect


On Dec 7, 2011, at 5:55 AM, Andrew Purtell <apurtell@yahoo.com> wrote:

> I need to review this code. 
> 
> That said, why couldn't this be handled by a postOpen up all?
> 
> Best regards,
> 
>    - Andy
> 
> 
> On Dec 7, 2011, at 5:24 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> 
>> Modified subject to match future discussion.
>> 
>> Andy:
>> In the latest patch for HBASE-4120, HRegionServer.addToOnlineRegions() is
>> used to refresh region priority.
>> Do you think RegionCoprocessorHost should be enhanced with more hooks, such
>> as preAddToOnlineRegions() ?
>> 
>> Thanks
>> 
>> On Tue, Dec 6, 2011 at 12:50 PM, Andrew Purtell <apurtell@yahoo.com> wrote:
>> 
>>> Also. I was going to start separate threads but probably there is not that
>>> much to discuss now or off-JIRA:
>>> 
>>> - I spoke with Jia Liu about HBASE-4120. After 0.92 is out I intend to
>>> champion / mentor / co-develop 4120 and the follow on table allocation work
>>> and target 0.94 for it. I think the RPC QoS aspect is not too
>>> controversial. The allocation/reservation aspects I'd like to aim for a
>>> coprocessor or at least master plugin based integration so they won't
>>> impact stability for users who don't enable it. Unlike RPC QoS I suspect
>>> the changes needed to core can be minimized to coprocessor framework
>>> additions. Follow up in new JIRAs soon.
>>> 
>>> - I also met with the author of "CCIndex". He is still interested in
>>> pursuing an upstreaming of this but may not have a lot of free time. The
>>> current implementation powers the data cube product at Taobao with half a
>>> million users IIRC, but the implementation is an invasive hack of the 0.20
>>> core code. I think the vast majority of it is an application layer concern
>>> that can be moved to a coprocessor implementation.  The coprocessor
>>> framework will need some modification as well, specifically hooks for
>>> handling IOEs taken on HFile reads.   Anyway I'm going to play around with
>>> this and follow up with a JIRA.
>>> 
>>> Best regards,
>>> 
>>>  - Andy
>>> 
>>> 
>>> On Dec 7, 2011, at 4:00 AM, Andrew Purtell <apurtell@apache.org> wrote:
>>> 
>>>> I also attended day 1 of Hadoop in China and would add these notes:
>>>> 
>>>> - Taobao has their own HA NameNode and is interested in contributing
>>> their changes upstream. I spoke with a platform dev lead. I told him do so
>>> now while HA is under active development rather than wait and miss an
>>> opportunity to participate/contribute. I need to follow up with him with a
>>> list of active HA jiras.
>>>> 
>>>> - Mingjie Lai presented on HBase and ZooKeeper security. I'll ask him to
>>> upload his slides somewhere.
>>>> 
>>>> 
>>>> Best regards,
>>>> 
>>>> 
>>>>      - Andy
>>>> 
>>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>>> 
>>>> 
>>>> ----- Original Message -----
>>>>> From: Todd Lipcon <todd@cloudera.com>
>>>>> To: dev@hbase.apache.org
>>>>> Cc:
>>>>> Sent: Wednesday, December 7, 2011 2:59 AM
>>>>> Subject: Re: trip report for Hadoop In China
>>>>> 
>>>>> Ah, I clicked on "hadooper.cn" directly instead of the earlier link to
>>>>> the agenda. Oops :)
>>>>> 
>>>>> Maybe you can put them in a public S3 bucket?
>>>>> 
>>>>> -Todd
>>>>> 
>>>>> On Tue, Dec 6, 2011 at 10:56 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>>> From http://hic2011.hadooper.cn/dct/page/1 (the first two lines):
>>>>>> Slides would be posted after getting permission from presenters
>>>>>> 
>>>>>> I can share my slides. My attempt at uploading ppt to slideshare
>>> failed and
>>>>>> I don't have PRO account.
>>>>>> 
>>>>>> I think apache.org would filter attachment.
>>>>>> 
>>>>>> Suggestion on where I can put my slides would be appreciated.
>>>>>> 
>>>>>> On Tue, Dec 6, 2011 at 10:39 AM, Todd Lipcon <todd@cloudera.com>
>>>>> wrote:
>>>>>> 
>>>>>>> Hey Ted,
>>>>>>> 
>>>>>>> Thanks for the trip report. Interesting stuff.
>>>>>>> 
>>>>>>> For those of us who can't read mandarin, can you link directly
to
>>>>> any
>>>>>>> english slide decks? The hadooper.cn link seems to go to an empty
>>> wiki
>>>>>>> template page or something, if I understand Google Translate
correctly
>>>>>>> :)
>>>>>>> 
>>>>>>> -Todd
>>>>>>> 
>>>>>>> On Tue, Dec 6, 2011 at 10:36 AM, Ted Yu <yuzhihong@gmail.com>
>>>>> wrote:
>>>>>>>> Hi,
>>>>>>>> There were around 1000 attendees for the first day of HiC.
>>>>>>>> The enthusiasm for cloud computing, hadoop in particular,
was
>>>>>>> overwhelming.
>>>>>>>> 
>>>>>>>> Doug Cutting's keynote was well received by attendees.
>>>>>>>> **Grzegorz's presentation about latest development @ Google,
>>>>> Pregel in
>>>>>>>> particular, was impressive.
>>>>>>>> Xiaodong Zhang's keynote about DOT, an analytical model,
was
>>>>> interesting.
>>>>>>>> 
>>>>>>>> I listened to several presentations. Here're a few:
>>>>>>>> 
>>>>>>>> Zheng Shao talked about Puma3 architecture which his group
>>>>> developed at
>>>>>>>> Facebook
>>>>>>>> Jerry and Liyin talked about how Facebook messaging uses
Hbase.
>>>>> Jerry
>>>>>>>> covered some best practices in using Hbase. Liyin talked
about
>>>>> some
>>>>>>>> features developed by their team, including Hfile V2, new
bloom
>>>>> filters,
>>>>>>> etc
>>>>>>>> Jinglong Liu from Baidu talked about how his team implemented
>>>>> compression
>>>>>>>> in hdfs (this is not RAID).
>>>>>>>> 
>>>>>>>> I met with developers / managers from Taobao, Huawei, Renren
>>>>> (Chinese
>>>>>>>> version of Facebook), etc and shared experience using HBase.
>>>>>>>> 
>>>>>>>> You can find the complete listing of talks here:
>>>>>>>> http://hic2011.hadooper.cn/dct/page/65580
>>>>>>>> 
>>>>>>>> On my way back, I met Doug Cutting at the airport.
>>>>>>>> He endorsed the idea of promoting committers from China so
that
>>>>> hadoop
>>>>>>>> community would have organic growth.
>>>>>>>> 
>>>>>>>> I didn't take detailed notes. So please find the slides from
>>>>> hadooper.cn
>>>>>>>> 
>>>>>>>> Thanks
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Todd Lipcon
>>>>>>> Software Engineer, Cloudera
>>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>>>> 
>>> 

Mime
View raw message