Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=DKIM-Signature:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Subject:References:From:Content-Type:X-Mailer:In-Reply-To:Message-Id:Date:To:Content-Transfer-Encoding:Mime-Version;
  b=OJqQPqwmTrTyc3HJPdmuoPgzgynelkZHaay1ztlR37063hDOxm+h90zKFpgMxoGRK3NenVQh2jR4nHf6Z0mT1psCuSz6U269C8G+3rwUEtw9T0rt04iBZuhpvx6f1IriwoJ0+eF3xu4MZbOmOiLC8PHOlaCp8PgGAU+ShfjbOSU=
  ;
Subject: Re: trip report for Hadoop In China
References: 
 <CALte62xg9Rg8z6XsjwUu4mANd2ckyxDTcaxMm9OQJtMkgTWhGw@mail.gmail.com>
 <CADY20s6zHcmhHjZ4iN9e3UxO7Mk4o1eLyvA6q6ZmAvRJ6kL9Qw@mail.gmail.com>
 <CALte62xPRj-8Z70fJ0+cBG3me+gFDTN9z=xdLzAvtK6s350iwg@mail.gmail.com>
 <CADY20s73prChsrrbobLRkn1Y5SHBPZUtGKRwKUtePyv7-NXLnA@mail.gmail.com>
 <1323201641.78858.YahooMailNeo@web65508.mail.ac4.yahoo.com>
From: Andrew Purtell <apurtell@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
In-Reply-To: <1323201641.78858.YahooMailNeo@web65508.mail.ac4.yahoo.com>
Message-Id: <92F38791-0C89-4C95-8258-52CC7F5DB25E@yahoo.com>
Date: Wed, 7 Dec 2011 04:50:20 +0800
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)

Also. I was going to start separate threads but probably there is not that m=
uch to discuss now or off-JIRA:

- I spoke with Jia Liu about HBASE-4120. After 0.92 is out I intend to champ=
ion / mentor / co-develop 4120 and the follow on table allocation work and t=
arget 0.94 for it. I think the RPC QoS aspect is not too controversial. The a=
llocation/reservation aspects I'd like to aim for a coprocessor or at least m=
aster plugin based integration so they won't impact stability for users who d=
on't enable it. Unlike RPC QoS I suspect the changes needed to core can be m=
inimized to coprocessor framework additions. Follow up in new JIRAs soon.=20=


- I also met with the author of "CCIndex". He is still interested in pursuin=
g an upstreaming of this but may not have a lot of free time. The current im=
plementation powers the data cube product at Taobao with half a million user=
s IIRC, but the implementation is an invasive hack of the 0.20 core code. I t=
hink the vast majority of it is an application layer concern that can be mov=
ed to a coprocessor implementation.  The coprocessor framework will need som=
e modification as well, specifically hooks for handling IOEs taken on HFile r=
eads.   Anyway I'm going to play around with this and follow up with a JIRA.=
=20

Best regards,

    - Andy


On Dec 7, 2011, at 4:00 AM, Andrew Purtell <apurtell@apache.org> wrote:

> I also attended day 1 of Hadoop in China and would add these notes:
>=20
> - Taobao has their own HA NameNode and is interested in contributing their=
 changes upstream. I spoke with a platform dev lead. I told him do so now wh=
ile HA is under active development rather than wait and miss an opportunity t=
o participate/contribute. I need to follow up with him with a list of active=
 HA jiras.
>=20
> - Mingjie Lai presented on HBase and ZooKeeper security. I'll ask him to u=
pload his slides somewhere.
>=20
> =20
> Best regards,
>=20
>=20
>        - Andy
>=20
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (=
via Tom White)
>=20
>=20
> ----- Original Message -----
>> From: Todd Lipcon <todd@cloudera.com>
>> To: dev@hbase.apache.org
>> Cc:=20
>> Sent: Wednesday, December 7, 2011 2:59 AM
>> Subject: Re: trip report for Hadoop In China
>>=20
>> Ah, I clicked on "hadooper.cn" directly instead of the earlier link to
>> the agenda. Oops :)
>>=20
>> Maybe you can put them in a public S3 bucket?
>>=20
>> -Todd
>>=20
>> On Tue, Dec 6, 2011 at 10:56 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> =46rom http://hic2011.hadooper.cn/dct/page/1 (the first two lines):
>>> Slides would be posted after getting permission from presenters
>>>=20
>>> I can share my slides. My attempt at uploading ppt to slideshare failed a=
nd
>>> I don't have PRO account.
>>>=20
>>> I think apache.org would filter attachment.
>>>=20
>>> Suggestion on where I can put my slides would be appreciated.
>>>=20
>>> On Tue, Dec 6, 2011 at 10:39 AM, Todd Lipcon <todd@cloudera.com>=20
>> wrote:
>>>=20
>>>> Hey Ted,
>>>>=20
>>>> Thanks for the trip report. Interesting stuff.
>>>>=20
>>>> For those of us who can't read mandarin, can you link directly to=20
>> any
>>>> english slide decks? The hadooper.cn link seems to go to an empty wiki
>>>> template page or something, if I understand Google Translate correctly
>>>> :)
>>>>=20
>>>> -Todd
>>>>=20
>>>> On Tue, Dec 6, 2011 at 10:36 AM, Ted Yu <yuzhihong@gmail.com>=20
>> wrote:
>>>>> Hi,
>>>>> There were around 1000 attendees for the first day of HiC.
>>>>> The enthusiasm for cloud computing, hadoop in particular, was
>>>> overwhelming.
>>>>>=20
>>>>> Doug Cutting's keynote was well received by attendees.
>>>>> **Grzegorz's presentation about latest development @ Google,=20
>> Pregel in
>>>>> particular, was impressive.
>>>>> Xiaodong Zhang's keynote about DOT, an analytical model, was=20
>> interesting.
>>>>>=20
>>>>> I listened to several presentations. Here're a few:
>>>>>=20
>>>>> Zheng Shao talked about Puma3 architecture which his group=20
>> developed at
>>>>> Facebook
>>>>> Jerry and Liyin talked about how Facebook messaging uses Hbase.=20
>> Jerry
>>>>> covered some best practices in using Hbase. Liyin talked about=20
>> some
>>>>> features developed by their team, including Hfile V2, new bloom=20
>> filters,
>>>> etc
>>>>> Jinglong Liu from Baidu talked about how his team implemented=20
>> compression
>>>>> in hdfs (this is not RAID).
>>>>>=20
>>>>> I met with developers / managers from Taobao, Huawei, Renren=20
>> (Chinese
>>>>> version of Facebook), etc and shared experience using HBase.
>>>>>=20
>>>>> You can find the complete listing of talks here:
>>>>> http://hic2011.hadooper.cn/dct/page/65580
>>>>>=20
>>>>> On my way back, I met Doug Cutting at the airport.
>>>>> He endorsed the idea of promoting committers from China so that=20
>> hadoop
>>>>> community would have organic growth.
>>>>>=20
>>>>> I didn't take detailed notes. So please find the slides from=20
>> hadooper.cn
>>>>>=20
>>>>> Thanks
>>>>=20
>>>>=20
>>>>=20
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>=20
>>=20
>>=20
>>=20
>> --=20
>> Todd Lipcon
>> Software Engineer, Cloudera
>>=20