hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Hu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-18879) Hbase FilterList cause KeyOnlyFilter not work
Date Thu, 12 Oct 2017 02:39:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zheng Hu updated HBASE-18879:
-----------------------------
    Attachment: HBASE-18879-HBASE-18410.v1.patch

> Hbase FilterList cause KeyOnlyFilter not work
> ---------------------------------------------
>
>                 Key: HBASE-18879
>                 URL: https://issues.apache.org/jira/browse/HBASE-18879
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Filters
>    Affects Versions: 1.2.4
>         Environment: OS: Red Hat 4.4.7-11
> Hadoop: 2.6.4
> Hbase: 1.2.4
>            Reporter: ZHA_Moonlight
>         Attachments: HBASE-18879-HBASE-18410.v1.patch
>
>
> when use FilterList and KeyOnlyFilter together, if we put KeyOnlyFilter before FilterList,
the KeyOnlyFilter may not work, means it will also grab the cell values:
> {code:java}
> List<Filter> filters = new ArrayList<Filter>();
>         Filter filter1 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"),
>                 CompareOp.EQUAL, Bytes.toBytes("value1"));
>         Filter filter2 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"),
>                 CompareOp.EQUAL, Bytes.toBytes("value2"));
>         filters.add(filter1);
>         filters.add(filter2);
>         FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL, 
>                 new KeyOnlyFilter(),
>                 new FilterList(Operator.MUST_PASS_ONE, filters));
> {code}
> use the above code as filter to scan a table, it will return the cells with value instead
of only return the key,  if we put KeyOnlyFilter after FilterList as following, it works well.
>           
> {code:java}
> FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL,
>                 new FilterList(Operator.MUST_PASS_ONE, filters),
>                 new KeyOnlyFilter());
> {code}
> the cause should due to the following code at hbase-client  FilterList.java
> {code:java}
> @Override
>   @edu.umd.cs.findbugs.annotations.SuppressWarnings(value="SF_SWITCH_FALLTHROUGH",
>     justification="Intentional")
>   public ReturnCode filterKeyValue(Cell v) throws IOException {
>     this.referenceKV = v;
>     // Accumulates successive transformation of every filter that includes the Cell:
>     Cell transformed = v;
>     ReturnCode rc = operator == Operator.MUST_PASS_ONE?
>         ReturnCode.SKIP: ReturnCode.INCLUDE;
>     int listize = filters.size();
>     for (int i = 0; i < listize; i++) {
>       Filter filter = filters.get(i);
>       if (operator == Operator.MUST_PASS_ALL) {
>         if (filter.filterAllRemaining()) {
>           return ReturnCode.NEXT_ROW;
>         }
> LINE1      ReturnCode code = filter.filterKeyValue(v);{color}        
> switch (code) {
>         // Override INCLUDE and continue to evaluate.
>         case INCLUDE_AND_NEXT_COL:
>           rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH
>         case INCLUDE:
> LINE2          transformed = filter.transformCell(transformed);{color}          
> continue;
>         case SEEK_NEXT_USING_HINT:
>           seekHintFilter = filter;
>           return code;
>         default:
>           return code;
>         }
>       }
> {code}
> notice the “LINE1”,"LINE2" , first line is a recursive invocation, it will assign
a Cell results to the FilterList.transformedKV(we call it A), the results is from the FilterList
with 2 SingleColumnValueFilter, so  A with contains the cell value, while the second line
with return  A to the var transformed.
> back to the following loop, we can see the FilterList return results is var "transformed
" which will override in each loop, so the value is determined by the last filter, so the
order of KeyOnlyFilter will impact the results.
> {code:java}
>  Cell transformed = v;
>     ReturnCode rc = operator == Operator.MUST_PASS_ONE?
>         ReturnCode.SKIP: ReturnCode.INCLUDE;
>     int listize = filters.size();
>     for (int i = 0; i < listize; i++) {
>       Filter filter = filters.get(i);
>       if (operator == Operator.MUST_PASS_ALL) {
>         if (filter.filterAllRemaining()) {
>           return ReturnCode.NEXT_ROW;
>         }
>         ReturnCode code = filter.filterKeyValue(v);
>         switch (code) {
>         // Override INCLUDE and continue to evaluate.
>         case INCLUDE_AND_NEXT_COL:
>           rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH
>         case INCLUDE:
>           transformed = filter.transformCell(transformed);
>           continue;
>         case SEEK_NEXT_USING_HINT:
>           seekHintFilter = filter;
>           return code;
>         default:
>           return code;
>         }
>       
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message