hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Re: How to define a custom filter to skip some amount of rows?
Date Fri, 17 Feb 2012 19:30:24 GMT
Filters support re-seek functionality. Hint can be given on the point
of re-seek. Re-seek can be applied to either columns or rows
(basically, hint is the new target of re-seek as key-value). You can
find details in the hbase book (and perhaps somewhere online, too).

-d

On Mon, Feb 13, 2012 at 5:27 PM, NNever <nneverwei@gmail.com> wrote:
> Hello~
> In HBase-0.92.0rc4.
> If I need to skip some amount of rows from the scan result, how can I
> define a custom filter to do it?
> Here is my solution:
>
> *public class SkipFilter extends FilterBase{*
> *    private long totalSkip = -1;*
> *    private long currentSkip = 0;*
> *    *
> *    public SkipFilter(){}*
> *
> *
> *    public SkipFilter(long totalSkip){*
> *        this.totalSkip = totalSkip;*
> *        currentSkip = 0;*
> *    }*
> *
> *
> *    // Only override this method*
> *    public boolean filterRowKey(byte[] buffer, int offset, int length){*
> *        if(currentSkip < totalSkip){*
> *                currentSkip++;*
> *                return true;*
> *        }else{*
> *                return false;*
> *        }*
> *    }*
> *
> *
> *    ...readFileds...*
> *    ...write...*
> *}*
>
> And used like this:
>
> *Scan scan = new Scan();*
> *scan.setStartRow(xxx);*
> *scan.setEndRow(xxx);*
> *Filter someFilter = new XXXFilter();*
> *Filter skipFilter = new SkipFilter(100);*
> *scan.setFilter(new FilterList(someFilter,...,skipFilter));*
> *...doScan....*
>
> *
> *
> Well, it works fine.  I scan some table, and the result can jump the first
> 100 rows, start from the 101th suitable row.
> But when I delete some row from this table, tragedy accure!! *The
> filter.filterRowKey can meet the deleted row*(I have post a mail before to
> discribe this).
> So if I try to skip the first 100 undeleted row, I user SkipFilter(100),
> but infact it can only filter out 99 rows.
> If I use this to do Paging, then at the second page I may meet the same row
> from first page.
>
> Can someone give me some suggestions on it or how to design a better
> SkipFilter to suit my need?
> Thanks!

Mime
View raw message