hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin <kevin.macksa...@gmail.com>
Subject Re: custom filter on hbase 0.96
Date Tue, 21 Oct 2014 18:03:43 GMT
BTW, the error looks like you didn't distribute your custom filter to your
region servers.

On Tue, Oct 21, 2014 at 1:34 PM, Kevin <kevin.macksamie@gmail.com> wrote:

> Matt,
>
> You should create your own proto file and compile that with the Google
> Protocol Buffer compiler. Take a look at the SingleColumnValueFilter's
> code:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java#L327
>
> You will need to override `public byte[] toByteArray()` and `public static
> Filter parseFrom(byte[] pbBytes)`. The output of toByteArray() should be
> the byte[] from serializing with the protocol buffer. This will also be the
> input to parseFrom(byte[]), which is called using reflection on the
> server-side to instantiate your custom filter and use it.
>
> On Sun, Oct 19, 2014 at 11:31 AM, Matt K <matvey1414@gmail.com> wrote:
>
>> Anyone?
>>
>> On Thursday, October 16, 2014, Matt K <matvey1414@gmail.com> wrote:
>>
>> > Hi, can anyone help with above? Feels like I'm missing something
>> obvious.
>> >
>> > On Wednesday, October 15, 2014, Nishanth S <nishanth.2884@gmail.com
>> > <javascript:_e(%7B%7D,'cvml','nishanth.2884@gmail.com');>> wrote:
>> >
>> >> Thanks Ted .I will take a look.
>> >>
>> >> -Nishanth
>> >>
>> >> On Wed, Oct 15, 2014 at 3:43 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >>
>> >> > Nishanth:
>> >> > Good question.
>> >> >
>> >> > As a general coding guide, writing unit test is always a good start.
>> >> Using
>> >> > Matt's case as an example, take a look at TestPrefixFilter.
>> >> >
>> >> > There're various unit tests for Filters in hbase code.
>> >> >
>> >> > Cheers
>> >> >
>> >> > On Wed, Oct 15, 2014 at 2:30 PM, Nishanth S <nishanth.2884@gmail.com
>> >
>> >> > wrote:
>> >> >
>> >> > > Hi Ted ,
>> >> > >  Since I am also working on similar thing is there a way we can
>> first
>> >> > test
>> >> > > the filter on client side?.You know what I  mean without disrupting
>> >> > others
>> >> > > who are using the same cluster for other work?
>> >> > >
>> >> > > Thanks,
>> >> > > Nishanth
>> >> > >
>> >> > > On Wed, Oct 15, 2014 at 3:17 PM, Ted Yu <yuzhihong@gmail.com>
>> wrote:
>> >> > >
>> >> > > > bq. Or create a new file, compile it into ...
>> >> > > >
>> >> > > > You should go with the above approach.
>> >> > > >
>> >> > > > On Wed, Oct 15, 2014 at 2:08 PM, Matt K <matvey1414@gmail.com>
>> >> wrote:
>> >> > > >
>> >> > > > > Hi all,
>> >> > > > >
>> >> > > > > I'm trying to get a custom filter to work on HBase 0.96.
After
>> >> some
>> >> > > > > searching, I found that starting from 0.96, the implementer
is
>> >> > required
>> >> > > > to
>> >> > > > > implement "toByteArray" and "parseFrom" methods, using
Protocol
>> >> > > Buffers.
>> >> > > > > But I'm having trouble with the "how".
>> >> > > > >
>> >> > > > > The "proto" file for the existing filters is located
here:
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://github.com/apache/hbase/blob/master/hbase-protocol/src/main/protobuf/Filter.proto
>> >> > > > >
>> >> > > > > Am I supposed to modify that file? Or create a new file,
>> compile
>> >> it
>> >> > > into
>> >> > > > > Java, and package it up with the filter?
>> >> > > > >
>> >> > > > > In the meantime, I've taken a shortcut that's not working.
>> Here's
>> >> my
>> >> > > > code:
>> >> > > > > http://pastebin.com/iHFKu9Xz
>> >> > > > >
>> >> > > > > I'm using "PrefixFilter", which comes with HBase, since
I'm
>> also
>> >> > > > filtering
>> >> > > > > by "prefix". However, that errors out with the following:
>> >> > > > > http://pastebin.com/zBg47p6Z
>> >> > > > >
>> >> > > > > Thanks in advance for helping!
>> >> > > > >
>> >> > > > > -Matt
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>> >
>> > --
>> > www.calcmachine.com - easy online calculator.
>> >
>>
>>
>> --
>> www.calcmachine.com - easy online calculator.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message