hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13347) RowCounter using special filter is broken
Date Wed, 28 Oct 2015 09:14:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978026#comment-14978026

Anoop Sam John commented on HBASE-13347:

Yes. As per the code the filterKeyValue() is been called after columns match.
// STEP 1: Check if the column is part of the requested columns
    MatchCode colChecker = columns.checkColumn(cell, typeByte);
    if (colChecker == MatchCode.INCLUDE) {
      ReturnCode filterResponse = ReturnCode.SKIP;
      // STEP 2: Yes, the column is part of the requested columns. Check if filter is present
      if (filter != null) {
        // STEP 3: Filter the key value and return if it filters out
        filterResponse = filter.filterKeyValue(cell);
So we actually don't want use FirstKeyValueMatchingQualifiersFilter as such.  May be we can
avoid that and always set FirstKeyOnlyFilter.

Can test with out any specific column add in Scan.  With one or more columns (cf:q).  With
one or more families added.

bq.As per the API doc "Only Filter.filterKeyValue(Cell) is called AFTER all tests for ttl,
column match, deletes and max versions have been run."
This is not fully true.  Version check happens after filter call.  Even TTL check also I guess
as one reported an issue in user@ related to that.

> RowCounter using special filter is broken
> -----------------------------------------
>                 Key: HBASE-13347
>                 URL: https://issues.apache.org/jira/browse/HBASE-13347
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 1.0.0
>            Reporter: Lars George
>             Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
> The {{RowCounter}} in the {{mapreduce}} package is supposed to check if the row count
scan has a column selection added to it, and if so, use a different filter that finds the
row and counts it. But the {{qualifier.add()}} call is missing in the {{for}} loop. See https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java#L165
> Needs fixing or row count might be wrong when using {{--range}}.

This message was sent by Atlassian JIRA

View raw message