hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Need hlep in substring filter
Date Mon, 20 Apr 2015 11:50:17 GMT
What's the value for hbase.dynamic.jars.dir ?

Cheers



> On Apr 20, 2015, at 12:53 AM, Jeetendra Gangele <gangele397@gmail.com> wrote:
> 
> I have put my jar in sub folder of hbase.rootdir still not working
> I am getting same error class not found.
> 
>> On 18 April 2015 at 03:54, Ted Yu <yuzhihong@gmail.com> wrote:
>> 
>> bq. This will @ hbase-site.xml?
>> 
>> Yes.
>> 
>> bq. can I pur my jar containing SuffixFilter class into hbase.rootdir?
>> 
>> Better create a sub-folder under the rootdir.
>> 
>> On Fri, Apr 17, 2015 at 1:58 PM, Jeetendra Gangele <gangele397@gmail.com>
>> wrote:
>> 
>>> This will @ hbase-site.xml?
>>> 
>>> can I pur my jar containing SuffixFilter class into hbase.rootdir?
>>> 
>>>> On 18 April 2015 at 02:19, Ted Yu <yuzhihong@gmail.com> wrote:
>>>> 
>>>> bq. Hbase 0.98.4 so this feature already should be there?
>>>> 
>>>> Yes.
>>>> 
>>>> Drop the class containing SuffixFilter class in location specified by
>>>> hbase.dynamic.jars.dir
>>>> config. Then region server should be able to find it.
>>>> 
>>>> On Fri, Apr 17, 2015 at 10:22 AM, Jeetendra Gangele <
>>> gangele397@gmail.com>
>>>> wrote:
>>>> 
>>>>> I am using Hbase 0.98.4 so this feature already should be there?
>>>>> problem is FilterList class(when we add this filter) not able to find
>>> the
>>>>> SuffixFilter
>>>>> 
>>>>>> On 17 April 2015 at 22:47, Jeetendra Gangele <gangele397@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Yes I used inside my project. This project will build the jar that
>>> jar
>>>> I
>>>>>> added with classpath when running the job
>>>>>> 
>>>>>>> On 17 April 2015 at 22:40, Ted Yu <yuzhihong@gmail.com>
wrote:
>>>>>>> 
>>>>>>> bq. java.io.IOException: java.lang.ClassNotFoundException:
>>>>>>> com.insideview.yam.HbaseClient.SuffixFilter
>>>>>>> 
>>>>>>> I assume you copied SuffixFilter class to your insideview project.
>>> How
>>>>> did
>>>>>>> you deploy the jar containing SuffixFilter to hbase cluster ?
>>>>>>> See HBASE-1936 for more details where hbase.dynamic.jars.dir
was
>>>>>>> introduced.
>>>>>>> 
>>>>>>> Cheers
>>>>>>> 
>>>>>>> On Fri, Apr 17, 2015 at 9:57 AM, Jeetendra Gangele <
>>>>> gangele397@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>>        at java.lang.Thread.run(Thread.java:745)
>>>>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>>>>        at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown
>>>>> Source)
>>>>>>>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>        at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>>        at
>> org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1439)
>>>>>>>>        ... 8 more
>>>>>>>> Caused by:
>>>>> org.apache.hadoop.hbase.exceptions.DeserializationException:
>>>>>>>> java.io.IOException: java.lang.ClassNotFoundException:
>>>>>>>> com.insideview.yam.HbaseClient.SuffixFilter
>>>>>>>>        at
>>> org.apache.hadoop.hbase.filter.FilterList.parseFrom(FilterList.java:406)
>>>>>>>>        ... 12 more
>>>>>>>> Caused by: java.io.IOException:
>> java.lang.ClassNotFoundException:
>>>>>>>> com.insideview.yam.HbaseClient.SuffixFilter
>>>>>>>>        at
>> org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1441)
>>>>>>>>        at
>>> org.apache.hadoop.hbase.filter.FilterList.parseFrom(FilterList.java:403)
>>>>>>>> 
>>>>>>>> This is the error i m getting
>>>>>>>>> On 17 April 2015 at 20:32, Ted Yu <yuzhihong@gmail.com>
wrote:
>>>>>>>>> 
>>>>>>>>> Here is an example
>>>>>>>>> from
>> phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
>>>>>>>>> :
>>>>>>>>> 
>>>>>>>>>        byte[] suffix =
>>>>>>>>> ByteUtil.concat(QueryConstants.SEPARATOR_BYTE_ARRAY,
>>>>>>>>> SchemaUtil.getTableNameAsBytes(schemaName, tableName));
>>>>>>>>>        SuffixFilter rowFilter = new SuffixFilter(suffix);
>>>>>>>>>        Filter filter = new FilterList(linkFilter, rowFilter);
>>>>>>>>> 
>>>>>>>>> You can use the following method from Bytes.java (in
>>> hbase-common)
>>>>> to
>>>>>>>>> obtain byte[] from "12345"
>>>>>>>>> 
>>>>>>>>>  public static byte[] toBytes(String s) {
>>>>>>>>> 
>>>>>>>>> Cheers
>>>>>>>>> 
>>>>>>>>> On Fri, Apr 17, 2015 at 7:50 AM, Jeetendra Gangele <
>>>>>>> gangele397@gmail.com
>>>>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> filterRowKey what is these arguments? Can you please
explain
>>>> lets
>>>>>>> say I
>>>>>>>>>> have data like abcde_12345 similary way
>>>>>>>>>> and I want to scan all the records ending with "12345"
what
>>>>> should I
>>>>>>>> pass
>>>>>>>>>> to this function?
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 16 April 2015 at 20:12, Ted Yu <yuzhihong@gmail.com>
>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Please take a look
>>>>>>>>>>> at
>> phoenix-core/src/main/java/org/apache/phoenix/coprocessor/SuffixFilter.java
>>>>>>>>>>> in Phoenix repo.
>>>>>>>>>>> 
>>>>>>>>>>> Processing is done in the following method:
>>>>>>>>>>> 
>>>>>>>>>>>  public boolean filterRowKey(byte[] buffer, int
offset,
>> int
>>>>>>> length)
>>>>>>>> {
>>>>>>>>>>> 
>>>>>>>>>>> Note: the suffix is expressed in byte array.
>>>>>>>>>>> 
>>>>>>>>>>> FYI
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Apr 16, 2015 at 7:27 AM, Jeetendra Gangele
<
>>>>>>>>> gangele397@gmail.com
>>>>>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Thanks for your reply.Ya I am filtering on
row-key which
>>>> ended
>>>>>>> with
>>>>>>>>>>> "12567"
>>>>>>>>>>>> 
>>>>>>>>>>>> I did not understand properly what do you
mean by "You
>> can
>>>>>>> compose
>>>>>>>>>>>> SuffixFilter which filters row key by comparing
suffix."
>>>>>>>>>>>> 
>>>>>>>>>>>> can you elaborate .
>>>>>>>>>>>> 
>>>>>>>>>>>> On 16 April 2015 at 19:49, Ted Yu <yuzhihong@gmail.com>
>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> bq. record key ending with "12567"
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Can you clarify whether you're filtering
by row key ?
>>>>>>>>>>>>> If so, please take a look at PrefixFilter.
>>>>>>>>>>>>> You can compose SuffixFilter which filters
row key by
>>>>>>> comparing
>>>>>>>>>> suffix.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Apr 16, 2015 at 7:06 AM, Jeetendra
Gangele <
>>>>>>>>>>> gangele397@gmail.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi All I have get all the records
using scan with
>> end
>>>> with
>>>>>>>>>> particular
>>>>>>>>>>>>>> string let
>>>>>>>>>>>>>> say record key ending with "12567".
I need to scan
>> and
>>>> get
>>>>>>> all
>>>>>>>>> such
>>>>>>>>>>>>> result.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I am values filter and do the substring
there but i
>>> want
>>>>> to
>>>>>>>> avoid
>>>>>>>>>>> this
>>>>>>>>>>>>>> because its slow.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> jeetendra
>> 

Mime
View raw message