Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.215.46 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CADQCNfdVQr3QoS7mWmUo2c51oqK7K65HOfM7SXiyguEhOg7SdA@mail.gmail.com>
References: 
 <CADQCNfdVQr3QoS7mWmUo2c51oqK7K65HOfM7SXiyguEhOg7SdA@mail.gmail.com>
Date: Tue, 20 Aug 2013 09:11:52 -0700
Message-ID: 
 <CALte62wY8mANpVY3VvBOzTxu5pZ_6yy4P3wCQZYcTOk86g_WEA@mail.gmail.com>
Subject: Re: Performance penalty: Custom Filter names serialization
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a11c36ad0a3adc804e46350a9

--001a11c36ad0a3adc804e46350a9
Content-Type: text/plain; charset=ISO-8859-1

Are you using HBase 0.92 or 0.94 ?

In 0.95 and later releases, HbaseObjectWritable doesn't exist. Protobuf is
used for communication.

Cheers


On Tue, Aug 20, 2013 at 8:56 AM, Pablo Medina <pablomedina85@gmail.com>wrote:

> Hi all,
>
> I'm using custom filters to retrieve filtered data from HBase using the
> native api. I noticed that the class full names of those custom filters is
> being sent as the bytes representation of the string using
> Text.writeString(). This consumes a lot of network bandwidth in my case due
> to using 5 custom filters per Get and issuing 1.5 million gets per minute.
> I took at look at the code (org.apache.hadoop.hbase.io.HbaseObjectWritable)
> and It seems that HBase registers its known classes (Get, Put, etc...) and
> associates them with an Integer (CODE_TO_CLASS and CLASS_TO_CODE). That
> integer is sent instead of the full class name for those known classes. I
> did a test reducing my custom filter class names to 2 or 3 letters and it
> improved my performance in 25%.
> Is there any way to "register" my custom filter classes to behave the same
> as HBase's classes? If not, does it make sense to introduce a change to do
> that? Is there any other workaround for this issue?
>
> Thanks!
>

--001a11c36ad0a3adc804e46350a9--