hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@yahoo.com>
Subject Re: trip report for Hadoop In China
Date Tue, 06 Dec 2011 20:50:20 GMT
Also. I was going to start separate threads but probably there is not that much to discuss
now or off-JIRA:

- I spoke with Jia Liu about HBASE-4120. After 0.92 is out I intend to champion / mentor /
co-develop 4120 and the follow on table allocation work and target 0.94 for it. I think the
RPC QoS aspect is not too controversial. The allocation/reservation aspects I'd like to aim
for a coprocessor or at least master plugin based integration so they won't impact stability
for users who don't enable it. Unlike RPC QoS I suspect the changes needed to core can be
minimized to coprocessor framework additions. Follow up in new JIRAs soon. 

- I also met with the author of "CCIndex". He is still interested in pursuing an upstreaming
of this but may not have a lot of free time. The current implementation powers the data cube
product at Taobao with half a million users IIRC, but the implementation is an invasive hack
of the 0.20 core code. I think the vast majority of it is an application layer concern that
can be moved to a coprocessor implementation.  The coprocessor framework will need some modification
as well, specifically hooks for handling IOEs taken on HFile reads.   Anyway I'm going to
play around with this and follow up with a JIRA. 

Best regards,

    - Andy

On Dec 7, 2011, at 4:00 AM, Andrew Purtell <apurtell@apache.org> wrote:

> I also attended day 1 of Hadoop in China and would add these notes:
> - Taobao has their own HA NameNode and is interested in contributing their changes upstream.
I spoke with a platform dev lead. I told him do so now while HA is under active development
rather than wait and miss an opportunity to participate/contribute. I need to follow up with
him with a list of active HA jiras.
> - Mingjie Lai presented on HBase and ZooKeeper security. I'll ask him to upload his slides
> Best regards,
>        - Andy
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
> ----- Original Message -----
>> From: Todd Lipcon <todd@cloudera.com>
>> To: dev@hbase.apache.org
>> Cc: 
>> Sent: Wednesday, December 7, 2011 2:59 AM
>> Subject: Re: trip report for Hadoop In China
>> Ah, I clicked on "hadooper.cn" directly instead of the earlier link to
>> the agenda. Oops :)
>> Maybe you can put them in a public S3 bucket?
>> -Todd
>> On Tue, Dec 6, 2011 at 10:56 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> From http://hic2011.hadooper.cn/dct/page/1 (the first two lines):
>>> Slides would be posted after getting permission from presenters
>>> I can share my slides. My attempt at uploading ppt to slideshare failed and
>>> I don't have PRO account.
>>> I think apache.org would filter attachment.
>>> Suggestion on where I can put my slides would be appreciated.
>>> On Tue, Dec 6, 2011 at 10:39 AM, Todd Lipcon <todd@cloudera.com> 
>> wrote:
>>>> Hey Ted,
>>>> Thanks for the trip report. Interesting stuff.
>>>> For those of us who can't read mandarin, can you link directly to 
>> any
>>>> english slide decks? The hadooper.cn link seems to go to an empty wiki
>>>> template page or something, if I understand Google Translate correctly
>>>> :)
>>>> -Todd
>>>> On Tue, Dec 6, 2011 at 10:36 AM, Ted Yu <yuzhihong@gmail.com> 
>> wrote:
>>>>> Hi,
>>>>> There were around 1000 attendees for the first day of HiC.
>>>>> The enthusiasm for cloud computing, hadoop in particular, was
>>>> overwhelming.
>>>>> Doug Cutting's keynote was well received by attendees.
>>>>> **Grzegorz's presentation about latest development @ Google, 
>> Pregel in
>>>>> particular, was impressive.
>>>>> Xiaodong Zhang's keynote about DOT, an analytical model, was 
>> interesting.
>>>>> I listened to several presentations. Here're a few:
>>>>> Zheng Shao talked about Puma3 architecture which his group 
>> developed at
>>>>> Facebook
>>>>> Jerry and Liyin talked about how Facebook messaging uses Hbase. 
>> Jerry
>>>>> covered some best practices in using Hbase. Liyin talked about 
>> some
>>>>> features developed by their team, including Hfile V2, new bloom 
>> filters,
>>>> etc
>>>>> Jinglong Liu from Baidu talked about how his team implemented 
>> compression
>>>>> in hdfs (this is not RAID).
>>>>> I met with developers / managers from Taobao, Huawei, Renren 
>> (Chinese
>>>>> version of Facebook), etc and shared experience using HBase.
>>>>> You can find the complete listing of talks here:
>>>>> http://hic2011.hadooper.cn/dct/page/65580
>>>>> On my way back, I met Doug Cutting at the airport.
>>>>> He endorsed the idea of promoting committers from China so that 
>> hadoop
>>>>> community would have organic growth.
>>>>> I didn't take detailed notes. So please find the slides from 
>> hadooper.cn
>>>>> Thanks
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>> -- 
>> Todd Lipcon
>> Software Engineer, Cloudera

View raw message