hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4176) Exposing HBase Filters to the Thrift API
Date Wed, 10 Aug 2011 03:01:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082113#comment-13082113
] 

jiraposter@reviews.apache.org commented on HBASE-4176:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1326/
-----------------------------------------------------------

(Updated 2011-08-10 03:01:21.623318)


Review request for hbase, Todd Lipcon, Ted Yu, Michael Stack, and Jonathan Gray.


Changes
-------

This update does two important things:

1. It enables the use of this Filter Language from the shell while keeping the existing functionality.
Below is some example usage (I can add more documentation to the document I had put up)

hbase(main):003:0> scan 't1', {FILTER => "KeyOnlyFilter()"}
ROW COLUMN+CELL

realtime column=conf:number, timestamp=1311055899185, value=
realtime column=conf:sameplesize, timestamp=1310962821624, value=
realtime column=conf:sameplesize, timestamp=1310962794886, value=
realtime column=conf:blacklist, timestamp=1310667339990, value=
realtime column=conf:daily_thresholds, timestamp=1310667461494, value=

hbase(main):002:0> scan 't1', {FILTER => org.apache.hadoop.hbase.filter.KeyOnlyFilter.new()}
ROW COLUMN+CELL

realtime column=conf:number, timestamp=1311055899185, value=
realtime column=conf:sameplesize, timestamp=1310962821624, value=
realtime column=conf:sameplesize, timestamp=1310962794886, value=
realtime column=conf:blacklist, timestamp=1310667339990, value=
realtime column=conf:daily_thresholds, timestamp=1310667461494, value=

hbase(main):005:0> scan 't1', {FILTER => "(FirstKeyOnlyFilter() AND ValueFilter(>=,
'binary:ghi')) OR TimestampsFilter(1311109736514)"}
ROW COLUMN+CELL
realtime column=conf:sameplesize, timestamp=1310970483721, value=nonononono
realtime column=conf:sameplesize, timestamp=1310962794886, value=nonononono
realtime column=conf:blacklist, timestamp=1310667339990, value=hello
row1 column=conf:number, timestamp=1311109736514, value=short_row

2. It reverts the changes to CompareFilter's LESS, LESS_OR_EQUAL etc. Upon testing the actual
results I got back (using the Filter Language from the shell) instead of just looking at whether
I was parsing and constructing the correct Filter objects - it seems this was incorrect b/c
of the way the code is structured. For example:

Suppose I am comparing a qualifier "abc" using a Qualifier Filter with a comparator EQUAL
and comparator - "abc" - then QualifierFilter's filterKeyValue calls doCompare. The compareResult
computed in doCompare evaluates to 0 b/c the two qualifiers are the same. It now enters the
EQUAL case since the compareOp was EQUASL. Here it checks that the compareResult is NOT_EQUAL
to 0 (instead of checking if it EQUAL to 0). Hence doCompare returns false (since they are
equal) and as a result in filterKeyValue we DON'T skip that key-value and instead INCLUDE
it

@Li - could you be more specific with what you mean when you say - "preconditions instead?"


Summary
-------

https://issues.apache.org/jira/browse/HBASE-4176: Exposing HBase Filters to the Thrift API

Currently, to use any of the filters, one has to explicitly add a scanner for the filter in
the Thrift API making it messy and long. 
With this patch, I am trying to add support for all the filters in a clean way. 
The user specifies a filter via a string. The string is parsed on the server to construct
the filter. More information can be found in the attached document named Filter Language

This patch is trying to extend and further the progress made by the patches in HBASE-1744

There is document attached to the HBASE-4176 JIRA that describes this patch in further detail


This addresses bug HBASE-4176.
    https://issues.apache.org/jira/browse/HBASE-4176


Diffs (updated)
-----

  /src/main/java/org/apache/hadoop/hbase/filter/ColumnCountGetFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/ColumnPrefixFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/ColumnRangeFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/CompareFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/DependentColumnFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/FamilyFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/Filter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/FilterBase.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/FilterList.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/FirstKeyOnlyFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/InclusiveStopFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/MultipleColumnPrefixFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/PageFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java PRE-CREATION 
  /src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java PRE-CREATION 
  /src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/QualifierFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/RowFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java 1155563

  /src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/TimestampsFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/filter/ValueFilter.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java 1155563 
  /src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java 1155563 
  /src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift 1155563 
  /src/main/ruby/hbase/table.rb 1155563 
  /src/main/ruby/shell/commands/scan.rb 1155563 
  /src/test/java/org/apache/hadoop/hbase/filter/TestParseFilter.java PRE-CREATION 

Diff: https://reviews.apache.org/r/1326/diff


Testing
-------

patch includes one test: TestParseFilter.java


Thanks,

Anirudh



> Exposing HBase Filters to the Thrift API
> ----------------------------------------
>
>                 Key: HBASE-4176
>                 URL: https://issues.apache.org/jira/browse/HBASE-4176
>             Project: HBase
>          Issue Type: Improvement
>          Components: thrift
>            Reporter: Anirudh Todi
>            Assignee: Anirudh Todi
>            Priority: Minor
>         Attachments: Filter Language(2).docx, Filter Language.docx, HBASE-4176.patch
>
>
> Currently, to use any of the filters, one has to explicitly add a scanner for the filter
in the Thrift API making it messy and long. With this patch, I am trying to add support for
all the filters in a clean way. The user specifies a filter via a string. The string is parsed
on the server to construct the filter. More information can be found in the attached document
named Filter Language
> This patch is trying to extend and further the progress made by the patches in the HBASE-1744
JIRA (https://issues.apache.org/jira/browse/HBASE-1744)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message