hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not next row?
Date Sat, 03 Mar 2012 04:43:41 GMT
German might have been easier, although I really meant "natural language" as opposed to code
:-)



________________________________
 From: "yuzhihong@gmail.com" <yuzhihong@gmail.com>
To: lars hofhansl <lhofhansl@yahoo.com> 
Cc: "user@hbase.apache.org" <user@hbase.apache.org> 
Sent: Friday, March 2, 2012 6:52 PM
Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but not
next row?
 

Thanks Lars for explaining this. 

Did you mean that expressing in German is easier :-)




On Mar 2, 2012, at 6:44 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:


I finally looked into it. This is expected.
>
>
>
>Filters are executed (by a StoreScanner/ScanQueryMatcher) per store. We have a store per
column family.
>The important part to observe here is that there is no intrinsic order between KeyValues
that only differ in the column family, that is by design so that stores can be handled in
parallel (even though we do not currently do that).
>
>
>The filter behaves as if every store is scanned in parallel. Each store starts in the
beginning, and then each store needs to skip ahead using the filter.
>This is why it seemed to you that NEXT_ROW only seeks to the next column family, because
you see the beginning of the scan for the next column family.
>
>
>
>Makes sense? It's a bit hard to explain in English :)
>
>
>
>-- Lars
>
>
>
>
>________________________________
> From: NNever <nneverwei@gmail.com>
>To: user@hbase.apache.org; Ted Yu <yuzhihong@gmail.com> 
>Sent: Wednesday, February 22, 2012 7:20 PM
>Subject: Re: HBase0.92: In Filter, ReturnCode.NEXT_ROW may lead to next columnFamily but
not next row?
> 
>Thanks Ted, I don't know mailing list strips attachment before.
>
>Here is the attache:
>
>TestFilter.java:    http://pastebin.com/zC6EF8pX
>and the log:   http://pastebin.com/RsKJSHcn
>
>2012/2/23 Ted Yu <yuzhihong@gmail.com>
>
>> N:
>> Can you publish your code on pastebin or somewhere ?
>> Mailing list strips attachment.
>>
>> Thanks
>>
>>
>> On Tue, Feb 21, 2012 at 5:47 PM, NNever <nneverwei@gmail.com> wrote:
>>
>>> Attach is my test customFilter code --- TestFilter.
>>> It just simply extends FilterBase and do some system.out...
>>> You can just try any Table has more than one columnFamily like below:
>>>
>>> *Scan scan = new Scan();*
>>> *scan.setFilter(new
 TestFilter());*
>>> *hTable.getScanner(scan);*
>>>
>>> and look the HBase's log...
>>>
>>> It seems there is truely a BUG here....When filterKeyValue return
>>> ReturnCode.NEXT_ROW, it jump to next columnFamily but not next row...
>>> also there is one thing strange, why the fitlerRow() not be called?
>>>
>>> 2012/2/21 <yuzhihong@gmail.com>
>>>
>>> The javadoc says filterRow() will still be called.
>>>>
>>>> Can you show us your filterRow() code ?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> On Feb 21, 2012, at 7:28 AM, NNever <nneverwei@gmail.com> wrote:
>>>>
>>>> > Hi~
>>>>
 >
>>>> > One customFilter,  Override filterKeyValue(KeyValue v).
>>>> > when the filter filterKeyValue a row's first keyValue, it will return
>>>> > "ReturnCode.NEXT_ROW" to jump to next row.
>>>> >
>>>> > But what infact is, the result changes when there are more than one
>>>> > columnFamily:(here are some logs)
>>>> >
>>>> > [filterRowKey] PERSONA1
>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> > [filterRowKey] PERSONA2
>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> > [filterRowKey] PERSONA3
>>>> > [filterKeyValue] family:info |
 qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> > [filterRowKey] PERSONA4
>>>> > [filterKeyValue] family:cert | qualifier:certSN | value:
>>>> > PERSONAL4314120472582094317514215676313826416149
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [filterKeyValue] family:info | qualifier:active | value:\x00
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [filterKeyValue] family:npo | qualifier:059201 | value:
>>>> > [filterKeyValue] returnCode is NEXT_ROW
>>>> > [reset]
>>>> >
>>>> > the Table schema is
>>>> > User
>>>> > info:name, info:address,
 info:active.... (info family, every record has
>>>> > values)
>>>> > npo:xxxx.....(npo family may be not exist)
>>>> > cert:certSN, cert:certType....(cert family may be not exist)
>>>> >
>>>> > As you can see from the log, once a row has more than one family
>>>> values,
>>>> > The ReturnCode.NEXT_ROW will not jump to next row but pass the left
>>>> > qualifier values and jump to next fmaily.
>>>> > Why? It's a bug or just design to be like this?
>>>> > Thanks~
>>>>
>>>
>>>
>>
>
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message