hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: HBase wire compatibility
Date Thu, 16 Feb 2012 20:48:26 GMT
Hi Devaraj,

IMO, the "protocol translator" interface introduced in HDFS has been
more pain than it's worth, given pluggability seems like a non-goal
for us. I think it's easier to just tie ourselves to protobufs. In
essence the client library (e.g. HTable) acts as the translator
between the user types and the wire types (protobuf).

That said, we should be absolutely sure that we don't expose protobufs
to the *client API*


On Thu, Feb 16, 2012 at 12:30 PM, Devaraj Das <ddas@hortonworks.com> wrote:
> Good writeup, Jimmy (was away for a few days due to an event in my family)
> Some quick questions - Has there been any thoughts on the plan to use HBASE-5394. Are
we going to make the hbase protocols (like HRegionInterface) protobuf aware?
> In Hadoop, I have seen the following:
> 1. In HDFS, the protocol definitions are not changed (like org.apache.hadoop.hdfs.protocol.ClientProtocol).
Instead there are translators that are defined that implement the mapping of protobuf datastructures
to application-level datastructures and vice versa (for example, have a look at ClientNamenodeProtocolTranslatorPB
and ClientNamenodeProtocolServerSideTranslatorPB in the package org.apache.hadoop.hdfs.protocolPB).
> 2. In Yarn (MRV2), all protocol definitions are written in PB
> Since the base RPC still uses writables for payload encoding, a translation happens when
the protobuf objects are sent/received (as an example look at org.apache.hadoop.ipc.ProtobufRpcEngine;
classes RpcRequestWritable and RpcResponseWritable).
> What does the HBase community think about the above?
> On Feb 13, 2012, at 1:02 PM, Jimmy Xiang wrote:
>> I posted the proposal on wiki:
>> http://wiki.apache.org/hadoop/Hbase/HBaseWireCompatibility
>> Thanks,
>> Jimmy
>> On Mon, Feb 13, 2012 at 11:03 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> Can you post on wiki ?
>>> Attachment stripped.
>>> On Mon, Feb 13, 2012 at 11:01 AM, Jimmy Xiang <jxiang@cloudera.com> wrote:
>>>> Hello,
>>>> As HBase installation base is getting bigger, we are ready to work on the
>>>> wire compatibility issue.
>>>> The goal is to make HBase easier for operators to upgrade, while it is
>>>> also easier for developers to
>>>> enhance, re-architect if necessary.
>>>> The attached is a proposal we came up.  We'd like to start with two
>>> phases:
>>>> Phase 1: Compatibility between client applications and HBase clusters
>>>> Phase 2: HBase cluster rolling upgrade within same major version
>>>> Could you please review?
>>>> Thanks,
>>>> Jimmy

Todd Lipcon
Software Engineer, Cloudera

View raw message