hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1652) Scanners for sparse column not stopped by StopRowFilter
Date Tue, 14 Jul 2009 15:59:14 GMT
Scanners for sparse column not stopped by StopRowFilter

                 Key: HBASE-1652
                 URL: https://issues.apache.org/jira/browse/HBASE-1652
             Project: Hadoop HBase
          Issue Type: Bug
          Components: filters, regionserver
    Affects Versions: 0.19.3
            Reporter: Dave Latham

Scanning a sparse column over a narrow range of rows can take far longer than expected because
the check for the end of the range is not performed on new rows unless there is a column match,
so it may end up scanning an entire region or table.

I have a table with 1 billion+ rows, and one cell in each row, generally small (10-1000 bytes).
 The columns are all in a single family and fairly sparse.  For one query, I run scans on
it to scan usually a narrow range of the table for the first 30 cells ina certain column.
 I know that all the rows that contain that column lie within a certain range.  I use HTable.getScanner(byte[][]
columns, byte[] startRow, RowFilterInterface filter) passing it the particular column I'm
looking for, a startRow, and a filter set containing a StopRowFilter wrapped in a WhileMatchRowFilter
to enforce the end of the range.  Sometimes the query is very fast (< 1 sec), but if the
table doesn't contain 30 rows with that column, it can be very slow, a minute or two.  I expected
that since the range was small, for example, just 120 rows, the query wouldn't take long to
scan the rows.

After some pondering and perusing of the source code, I think I understand what is going on.
 It looks like the Scanner is scanning the rest of the table to find rows containing the column
without allowing the StopRowFilter to stop the scan at the end of the range.  I think I can
work around this by not specifying the column I want in the getScanner() method and instead
putting an additional filter in the filter set to filter out other columns.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message