Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 26 Sep 2017 11:17:00 +0000 (UTC)
From: "Zheng Hu (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13104978.1506397971000.203268.1506424620132@Atlassian.JIRA>
In-Reply-To: <JIRA.13104978.1506397971000@Atlassian.JIRA>
References: <JIRA.13104978.1506397971000@Atlassian.JIRA> <JIRA.13104978.1506397971763@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HBASE-18879) Hbase FilterList cause
 KeyOnlyFilter not work
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 26 Sep 2017 11:17:05 -0000


    [ https://issues.apache.org/jira/browse/HBASE-18879?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D161=
80615#comment-16180615 ]=20

Zheng Hu commented on HBASE-18879:
----------------------------------

Thanks for the debug.  It seems like a BUG.  =20

For FilterList, we can consider a filter list as a node in a tree.  childre=
n of the node are the sub-filters in the relative filter list.   The logic =
of transforming cell of a filter list , well,  we can consider it as the pr=
ocess of  post-order tree traverse.  For a node ,  Before we traverse the c=
urrent child,  we should set the traverse result (transformed cell) of prev=
ious node(s) as the initial value.   So I think the problem is here:=20

{code}
// Accumulates successive transformation of every filter that includes the =
Cell:
Cell transformed =3D v;     -------------------------->  should set the tra=
verse result of previous node(s) as the initial value(v if there's no previ=
ous node ),  not the v.=20
ReturnCode rc =3D operator =3D=3D Operator.MUST_PASS_ONE?
ReturnCode.SKIP: ReturnCode.INCLUDE;
{code}

The  pseudo-code should be :=20
{code}
ReturnCode filterKeyValue(v, initialTransformKV){
    transformKV =3D initialTransformKV =3D=3D null ? v: initialTransformKV;
    for(int i =3D 0 ; i < filters.size(); i++){
       filterKeyValue(v, transformKV);
       transformKV =3D filters[i].transformCell(transformKV);
       // ...merge return code...=20
    }
    return rc;=20
}

ReturnCode filterKeyValue(v){
    return filterKeyValue(v, null);
}
{code}

[~pengxu] , [~anoop.hbase], [~Apache9] , Could you take a look ?=20

> Hbase FilterList cause KeyOnlyFilter not work
> ---------------------------------------------
>
>                 Key: HBASE-18879
>                 URL: https://issues.apache.org/jira/browse/HBASE-18879
>             Project: HBase
>          Issue Type: Bug
>          Components: Filters
>    Affects Versions: 1.2.4
>         Environment: OS=EF=BC=9A Red Hat 4.4.7-11
> Hadoop: 2.6.4
> Hbase: 1.2.4
>            Reporter: ZHA_Moonlight
>
> when use FilterList and KeyOnlyFilter together, if we put KeyOnlyFilter b=
efore FilterList, the KeyOnlyFilter may not work, means it will also grab t=
he cell values:
> {code:java}
> List<Filter> filters =3D new ArrayList<Filter>();
>         Filter filter1 =3D new SingleColumnValueFilter(Bytes.toBytes("cf"=
), Bytes.toBytes("column1"),
>                 CompareOp.EQUAL, Bytes.toBytes("value1"));
>         Filter filter2 =3D new SingleColumnValueFilter(Bytes.toBytes("cf"=
), Bytes.toBytes("column1"),
>                 CompareOp.EQUAL, Bytes.toBytes("value2"));
>         filters.add(filter1);
>         filters.add(filter2);
>         FilterList filterListAll =3D new FilterList(Operator.MUST_PASS_AL=
L,=20
>                 new KeyOnlyFilter(),
>                 new FilterList(Operator.MUST_PASS_ONE, filters));
> {code}
> use the above code as filter to scan a table, it will return the cells wi=
th value instead of only return the key,  if we put KeyOnlyFilter after Fil=
terList as following, it works well.
>          =20
> {code:java}
> FilterList filterListAll =3D new FilterList(Operator.MUST_PASS_ALL,
>                 new FilterList(Operator.MUST_PASS_ONE, filters),
>                 new KeyOnlyFilter());
> {code}
> the cause should due to the following code at hbase-client  FilterList.ja=
va
> {code:java}
> @Override
>   @edu.umd.cs.findbugs.annotations.SuppressWarnings(value=3D"SF_SWITCH_FA=
LLTHROUGH",
>     justification=3D"Intentional")
>   public ReturnCode filterKeyValue(Cell v) throws IOException {
>     this.referenceKV =3D v;
>     // Accumulates successive transformation of every filter that include=
s the Cell:
>     Cell transformed =3D v;
>     ReturnCode rc =3D operator =3D=3D Operator.MUST_PASS_ONE?
>         ReturnCode.SKIP: ReturnCode.INCLUDE;
>     int listize =3D filters.size();
>     for (int i =3D 0; i < listize; i++) {
>       Filter filter =3D filters.get(i);
>       if (operator =3D=3D Operator.MUST_PASS_ALL) {
>         if (filter.filterAllRemaining()) {
>           return ReturnCode.NEXT_ROW;
>         }
> LINE1      ReturnCode code =3D filter.filterKeyValue(v);{color}       =20
> switch (code) {
>         // Override INCLUDE and continue to evaluate.
>         case INCLUDE_AND_NEXT_COL:
>           rc =3D ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_F=
ALLTHROUGH
>         case INCLUDE:
> LINE2          transformed =3D filter.transformCell(transformed);{color} =
        =20
> continue;
>         case SEEK_NEXT_USING_HINT:
>           seekHintFilter =3D filter;
>           return code;
>         default:
>           return code;
>         }
>       }
> {code}
> notice the =E2=80=9CLINE1=E2=80=9D,"LINE2" , first line is a recursive in=
vocation, it will assign a Cell results to the FilterList.transformedKV(we =
call it A), the results is from the FilterList with 2 SingleColumnValueFilt=
er, so  A with contains the cell value, while the second line with return  =
A to the var transformed.
> back to the following loop, we can see the FilterList return results is v=
ar "transformed " which will override in each loop, so the value is determi=
ned by the last filter, so the order of KeyOnlyFilter will impact the resul=
ts.
> {code:java}
>  Cell transformed =3D v;
>     ReturnCode rc =3D operator =3D=3D Operator.MUST_PASS_ONE?
>         ReturnCode.SKIP: ReturnCode.INCLUDE;
>     int listize =3D filters.size();
>     for (int i =3D 0; i < listize; i++) {
>       Filter filter =3D filters.get(i);
>       if (operator =3D=3D Operator.MUST_PASS_ALL) {
>         if (filter.filterAllRemaining()) {
>           return ReturnCode.NEXT_ROW;
>         }
>         ReturnCode code =3D filter.filterKeyValue(v);
>         switch (code) {
>         // Override INCLUDE and continue to evaluate.
>         case INCLUDE_AND_NEXT_COL:
>           rc =3D ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_F=
ALLTHROUGH
>         case INCLUDE:
>           transformed =3D filter.transformCell(transformed);
>           continue;
>         case SEEK_NEXT_USING_HINT:
>           seekHintFilter =3D filter;
>           return code;
>         default:
>           return code;
>         }
>      =20
> {code}


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)