hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18160) Fix incorrect logic in FilterList.filterKeyValue
Date Fri, 29 Sep 2017 23:35:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186685#comment-16186685
] 

Ted Yu commented on HBASE-18160:
--------------------------------

Running with branch-1.3 patch, I got:
{code}
Test(org.apache.hadoop.hbase.filter.TestFuzzyRowAndColumnRangeFilter)  Time elapsed: 2.83
sec  <<< FAILURE!
java.lang.AssertionError: expected:<8> but was:<0>
	at org.apache.hadoop.hbase.filter.TestFuzzyRowAndColumnRangeFilter.runScanner(TestFuzzyRowAndColumnRangeFilter.java:181)
	at org.apache.hadoop.hbase.filter.TestFuzzyRowAndColumnRangeFilter.runTest(TestFuzzyRowAndColumnRangeFilter.java:152)
	at org.apache.hadoop.hbase.filter.TestFuzzyRowAndColumnRangeFilter.Test(TestFuzzyRowAndColumnRangeFilter.java:132)

Running org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd
Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 97.124 sec <<< FAILURE!
- in org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd
testFilterList(org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd)  Time elapsed: 3.095
sec  <<< ERROR!
org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException:
was there a rpc timeout?
	at org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd.runScanner(TestFuzzyRowFilterEndToEnd.java:430)
	at org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd.runTest(TestFuzzyRowFilterEndToEnd.java:415)
	at org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd.testFilterList(TestFuzzyRowFilterEndToEnd.java:383)
Caused by: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException:
org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 1
But the nextCallSeq got from client: 0; request=scanner_id: 7826462120291074081 number_of_rows:
1000 close_scanner: false next_call_seq: 0 client_handles_partials: true client_handles_heartbeats:
true track_scan_metrics: false renew: false
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkScanNextCallSeq(RSRpcServices.java:2573)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2849)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:35072)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2347)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)

Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException:
org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 1
But the nextCallSeq got from client: 0; request=scanner_id: 7826462120291074081 number_of_rows:
1000 close_scanner: false next_call_seq: 0 client_handles_partials: true client_handles_heartbeats:
true track_scan_metrics: false renew: false
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkScanNextCallSeq(RSRpcServices.java:2573)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2849)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:35072)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2347)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
{code}
I can provide test output if needed. But the test failure should be easy to reproduce.

> Fix incorrect  logic in FilterList.filterKeyValue
> -------------------------------------------------
>
>                 Key: HBASE-18160
>                 URL: https://issues.apache.org/jira/browse/HBASE-18160
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Zheng Hu
>            Assignee: Zheng Hu
>             Fix For: 1.4.0, 1.2.6, 1.3.2, 2.0.0-alpha-3
>
>         Attachments: filter-and-map.txt, filter-or-map.txt, HBASE-18160.branch-1.3.v1.patch,
HBASE-18160.branch-1.3.v1.patch, HBASE-18160.branch-1.4.v1.patch, HBASE-18160.branch-1.4.v1.patch,
HBASE-18160.v1.patch, HBASE-18160.v2.patch, HBASE-18160.v2.patch, HBASE-18160.v3.patch, HBASE-18160.v4.patch,
HBASE-18160.v5.patch, HBASE-18160.v6.patch, HBASE-18160.v6.patch, HBASE-18160.v7.patch, HBASE-18160.v8.patch
>
>
> As HBASE-17678 said, there are two problems in FilterList.filterKeyValue implementation:

> 1.  FilterList did not consider INCLUDE_AND_SEEK_NEXT_ROW case( seems like INCLUDE_AND_SEEK_NEXT_ROW
is a newly added case, and the dev forgot to consider FilterList), So if a user use INCLUDE_AND_SEEK_NEXT_ROW
in his own Filter and wrapped by a FilterList,  it'll  throw  an IllegalStateException("Received
code is not valid."). 
> 2.  For FilterList with MUST_PASS_ONE,   if filter-A in filter list return  INCLUDE and
filter-B in filter list return INCLUDE_AND_NEXT_COL,   the FilterList will return  INCLUDE_AND_NEXT_COL
finally.  According to the mininal step rule , It's incorrect.  (filter list with MUST_PASS_ONE
choose the mininal step among filters in filter list. Let's call it: The Mininal Step Rule).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message