hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "donhoff_h" <165612...@qq.com>
Subject 回复: HBase Filter Problem
Date Wed, 23 Sep 2015 09:01:17 GMT

There are 90 Million records in the table. And I use the the MUST_PASS_ALL for all my filters.
 When I use PrefixFilter + SingleColumnValueFilter, it returned fast. So I supposed that the
combination of PrefixFilter + SingleColumnValueFilter + FirstKeyOnlyFilter should be fast.
But the fact is just in contrast. Do you know the reason that cause it?


------------------ 原始邮件 ------------------
发件人: "Fulin Sun";<sunfl@certusnet.com.cn>;
发送时间: 2015年9月23日(星期三) 下午4:53
收件人: "HBase User"<user@hbase.apache.org>; 

主题: 回复: HBase Filter Problem

Hi , there

How many rows are there in the hbase table ? You want to achive the default FilterList.Operator.MUST_PASS_ALL
you just want to use or conditions for these filters ? 

I think the reason is that this kind of filter list just go more scan work and lower performance.



发件人: donhoff_h
发送时间: 2015-09-23 16:33
收件人: user
主题: HBase Filter Problem
I wrote a program which function is to extract some data from a HBase table. According to
business requirements I had to use the PrefixFilter and the SingleColumnValueFilter to filter
the data.  The program ran very fast and returned in 1 sec. 
Considering I just need the rowkey of each record in my final result, I tried to improve my
program by using the PrefixFilter + SingleColumnValueFilter + FirstKeyOnlyFitler. To my surprise
the program ran very slow this time. It run about 20min and still not finished. So I had to
kill it.
Does anybody know the reason that cause my program run such slow?  Since I set the PrefixFilter
as the first filter in the FilterList object, I think the program should ran fast.
Many Thanks!
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message