hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Need hlep in substring filter
Date Fri, 17 Apr 2015 22:24:37 GMT
bq. This will @ hbase-site.xml?

Yes.

bq. can I pur my jar containing SuffixFilter class into hbase.rootdir?

Better create a sub-folder under the rootdir.

On Fri, Apr 17, 2015 at 1:58 PM, Jeetendra Gangele <gangele397@gmail.com>
wrote:

> This will @ hbase-site.xml?
>
> can I pur my jar containing SuffixFilter class into hbase.rootdir?
>
> On 18 April 2015 at 02:19, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > bq. Hbase 0.98.4 so this feature already should be there?
> >
> > Yes.
> >
> > Drop the class containing SuffixFilter class in location specified by
> > hbase.dynamic.jars.dir
> > config. Then region server should be able to find it.
> >
> > On Fri, Apr 17, 2015 at 10:22 AM, Jeetendra Gangele <
> gangele397@gmail.com>
> > wrote:
> >
> > > I am using Hbase 0.98.4 so this feature already should be there?
> > > problem is FilterList class(when we add this filter) not able to find
> the
> > > SuffixFilter
> > >
> > > On 17 April 2015 at 22:47, Jeetendra Gangele <gangele397@gmail.com>
> > wrote:
> > >
> > > > Yes I used inside my project. This project will build the jar that
> jar
> > I
> > > > added with classpath when running the job
> > > >
> > > > On 17 April 2015 at 22:40, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > >> bq. java.io.IOException: java.lang.ClassNotFoundException:
> > > >> com.insideview.yam.HbaseClient.SuffixFilter
> > > >>
> > > >> I assume you copied SuffixFilter class to your insideview project.
> How
> > > did
> > > >> you deploy the jar containing SuffixFilter to hbase cluster ?
> > > >> See HBASE-1936 for more details where hbase.dynamic.jars.dir was
> > > >> introduced.
> > > >>
> > > >> Cheers
> > > >>
> > > >> On Fri, Apr 17, 2015 at 9:57 AM, Jeetendra Gangele <
> > > gangele397@gmail.com>
> > > >> wrote:
> > > >>
> > > >> >         at java.lang.Thread.run(Thread.java:745)
> > > >> > Caused by: java.lang.reflect.InvocationTargetException
> > > >> >         at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown
> > > Source)
> > > >> >         at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >> >         at java.lang.reflect.Method.invoke(Method.java:606)
> > > >> >         at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1439)
> > > >> >         ... 8 more
> > > >> > Caused by:
> > > org.apache.hadoop.hbase.exceptions.DeserializationException:
> > > >> > java.io.IOException: java.lang.ClassNotFoundException:
> > > >> > com.insideview.yam.HbaseClient.SuffixFilter
> > > >> >         at
> > > >> >
> > >
> org.apache.hadoop.hbase.filter.FilterList.parseFrom(FilterList.java:406)
> > > >> >         ... 12 more
> > > >> > Caused by: java.io.IOException: java.lang.ClassNotFoundException:
> > > >> > com.insideview.yam.HbaseClient.SuffixFilter
> > > >> >         at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1441)
> > > >> >         at
> > > >> >
> > >
> org.apache.hadoop.hbase.filter.FilterList.parseFrom(FilterList.java:403)
> > > >> >
> > > >> > This is the error i m getting
> > > >> > On 17 April 2015 at 20:32, Ted Yu <yuzhihong@gmail.com>
wrote:
> > > >> >
> > > >> > > Here is an example
> > > >> > > from
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
> > > >> > > :
> > > >> > >
> > > >> > >         byte[] suffix =
> > > >> > > ByteUtil.concat(QueryConstants.SEPARATOR_BYTE_ARRAY,
> > > >> > > SchemaUtil.getTableNameAsBytes(schemaName, tableName));
> > > >> > >         SuffixFilter rowFilter = new SuffixFilter(suffix);
> > > >> > >         Filter filter = new FilterList(linkFilter, rowFilter);
> > > >> > >
> > > >> > > You can use the following method from Bytes.java (in
> hbase-common)
> > > to
> > > >> > > obtain byte[] from "12345"
> > > >> > >
> > > >> > >   public static byte[] toBytes(String s) {
> > > >> > >
> > > >> > > Cheers
> > > >> > >
> > > >> > > On Fri, Apr 17, 2015 at 7:50 AM, Jeetendra Gangele <
> > > >> gangele397@gmail.com
> > > >> > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > filterRowKey what is these arguments? Can you please
explain
> > lets
> > > >> say I
> > > >> > > > have data like abcde_12345 similary way
> > > >> > > > and I want to scan all the records ending with "12345"
what
> > > should I
> > > >> > pass
> > > >> > > > to this function?
> > > >> > > >
> > > >> > > >
> > > >> > > > On 16 April 2015 at 20:12, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > > >> > > >
> > > >> > > > > Please take a look
> > > >> > > > > at
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> phoenix-core/src/main/java/org/apache/phoenix/coprocessor/SuffixFilter.java
> > > >> > > > > in Phoenix repo.
> > > >> > > > >
> > > >> > > > > Processing is done in the following method:
> > > >> > > > >
> > > >> > > > >   public boolean filterRowKey(byte[] buffer, int
offset, int
> > > >> length)
> > > >> > {
> > > >> > > > >
> > > >> > > > > Note: the suffix is expressed in byte array.
> > > >> > > > >
> > > >> > > > > FYI
> > > >> > > > >
> > > >> > > > > On Thu, Apr 16, 2015 at 7:27 AM, Jeetendra Gangele
<
> > > >> > > gangele397@gmail.com
> > > >> > > > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Thanks for your reply.Ya I am filtering on
row-key which
> > ended
> > > >> with
> > > >> > > > > "12567"
> > > >> > > > > >
> > > >> > > > > > I did not understand properly what do you
mean by "You can
> > > >> compose
> > > >> > > > > > SuffixFilter which filters row key by comparing
suffix."
> > > >> > > > > >
> > > >> > > > > > can you elaborate .
> > > >> > > > > >
> > > >> > > > > > On 16 April 2015 at 19:49, Ted Yu <yuzhihong@gmail.com>
> > > wrote:
> > > >> > > > > >
> > > >> > > > > > > bq. record key ending with "12567"
> > > >> > > > > > >
> > > >> > > > > > > Can you clarify whether you're filtering
by row key ?
> > > >> > > > > > > If so, please take a look at PrefixFilter.
> > > >> > > > > > > You can compose SuffixFilter which filters
row key by
> > > >> comparing
> > > >> > > > suffix.
> > > >> > > > > > >
> > > >> > > > > > > Cheers
> > > >> > > > > > >
> > > >> > > > > > > On Thu, Apr 16, 2015 at 7:06 AM, Jeetendra
Gangele <
> > > >> > > > > gangele397@gmail.com
> > > >> > > > > > >
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Hi All I have get all the records
using scan with end
> > with
> > > >> > > > particular
> > > >> > > > > > > > string let
> > > >> > > > > > > > say record key ending with "12567".
I need to scan and
> > get
> > > >> all
> > > >> > > such
> > > >> > > > > > > result.
> > > >> > > > > > > >
> > > >> > > > > > > > I am values filter and do the substring
there but i
> want
> > > to
> > > >> > avoid
> > > >> > > > > this
> > > >> > > > > > > > because its slow.
> > > >> > > > > > > >
> > > >> > > > > > > > Regards
> > > >> > > > > > > > jeetendra
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message