hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allan Yan <hailun...@gmail.com>
Subject Why RowFilter plus BinaryPrefixComparator solution is so slow
Date Thu, 11 Aug 2011 21:20:02 GMT
Hello,

We need to do range query from 20 million rows table. I thought using
RowFilter with BinaryPrefixComparator will help. It gets the result
correctly. However, the speed is not acceptable. Here is the code
snippet:

1. Scan s = new Scan();
2. s.addFamily(myFamily);
3. s.setStartRow(startRow);
4. Filter rowFilter = new RowFilter(CompareFilter.CompareOp.EQUAL, new
BinaryPrefixComparator(startRow));
5. s.setFilter(rowFilter);

It takes 23 seconds to return 1 record back to client.

If I comment out line 4 and 5 (meaning without filter), it only takes
400 milliseconds to return 10 rows (by calling
table.getScanner().next() 10 times).

In addition, If I don't use filter and simply set up startRow and
stopRow for the Scan object, the same search(return 1 record) only
takes 40 milliseconds.

Did I missed something in setting up filter? If this is not the right
way to use this RowFilter and BinaryPrefixComparator, how should I use
it?

Thanks,
Allan Yan

Mime
View raw message