hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (POWERSET)" <Jim.Keller...@microsoft.com>
Subject RE: RE: RE: who's doing what for 0.21?
Date Mon, 22 Jun 2009 16:23:21 GMT
> -----Original Message-----
> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
> Sent: Saturday, June 20, 2009 6:15 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Re: RE: RE: who's doing what for 0.21?
>
> I note none of those targets have performance as an explicit goal.
>
> Right now hadoop rpc is too low performing they end up using the
> xciever
> protocol...so will they replace that as well?  Using a json defined
> self
> describing auto negotiating no code gen system?

That is my understanding.

> If we want to consider using it, we need a 1st class seat at the
> table and
> have to push hard for what we need.
>
> Does anyone know the people involved personally?

Doug Cutting is leading the project. I don't know who else is
involved.

I think they recognize that the current RPC is slow and a
resource hog (xceivers, open file handles).

They also want to get rid of the raw socket protocol used
between clients and data nodes.

The goal overall however is language agnostic wire protocol
without the need to compile anything.

Finally, I agree that if it is not performant then we should
not use it.

> On Jun 20, 2009 6:04 PM, "Jim Kellerman (POWERSET)" <
> Jim.Kellerman@microsoft.com> wrote:
>
> +1 on performance testing
>
> but on a side note, when Avro is integrated into Hadoop, they
> are looking at replacing the RPC
> - to make it language agnostic
> - and to introduce non-blocking soket I/O
> - and *finally* to multiplex sockets, reducing the number of open
>  file handles.
>
> --- Jim Kellerman, Powerset (Live Search, Microsoft Corporation) >
> -----Original Message-----
>
> > From: Ryan Rawson [mailto:ryanobjc@gmail.com] > Sent: Saturday,
> June 20,
> 2009 3:48 PM > To: hbase-...
>
> > Subject: Re: RE: who's doing what for 0.21? > > Man I can't
> believe my
> email client let me send an...

Mime
View raw message