hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "donhoff_h" <165612...@qq.com>
Subject 回复: HBase Filter Problem
Date Fri, 25 Sep 2015 02:34:26 GMT

There are many other columns and one such column was used in the SingleColumnValueFilter.
My intention is first use the PrefixFilter to narrow the data scope, then use the SingleColumnValueFilter
to choose the correct record, and last use the FirstKeyOnlyFilter to get just one KV and extract
the rowkey from it. But the result shows that the SingleColumnValueFilter seems can not co-work
with the FirstKeyOnlyFilter.  So I want to get understood the mechanism that cause this.

------------------ 原始邮件 ------------------
发件人: "ramkrishna vasudevan";<ramkrishna.s.vasudevan@gmail.com>;
发送时间: 2015年9月24日(星期四) 下午3:21
收件人: "user@hbase.apache.org"<user@hbase.apache.org>; 

主题: Re: HBase Filter Problem

Just trying to understand  more,
you are having a combination of PRefixFilter and SingleColumnValueFilter -
now the column you have specified in the SingleColumnValueFilter -  is it
the only column that you have in your table?  Or is there many other
columns and one such column was used in the SingleColumnValueFilter?

The idea of FirstKeyOnlyFilter is just to skip to the next row on getting
the first ever column in that row.  May be the combination of these two is
causing some issues.


On Wed, Sep 23, 2015 at 2:31 PM, donhoff_h <165612158@qq.com> wrote:

> Hi,
> There are 90 Million records in the table. And I use the the MUST_PASS_ALL
> for all my filters.  When I use PrefixFilter + SingleColumnValueFilter, it
> returned fast. So I supposed that the combination of PrefixFilter +
> SingleColumnValueFilter + FirstKeyOnlyFilter should be fast. But the fact
> is just in contrast. Do you know the reason that cause it?
> Thanks!
> ------------------ 原始邮件 ------------------
> 发件人: "Fulin Sun";<sunfl@certusnet.com.cn>;
> 发送时间: 2015年9月23日(星期三) 下午4:53
> 收件人: "HBase User"<user@hbase.apache.org>;
> 主题: 回复: HBase Filter Problem
> Hi , there
> How many rows are there in the hbase table ? You want to achive the
> default FilterList.Operator.MUST_PASS_ALL or
> you just want to use or conditions for these filters ?
> I think the reason is that this kind of filter list just go more scan work
> and lower performance.
> Best,
> Sun.
> CertusNet
> 发件人: donhoff_h
> 发送时间: 2015-09-23 16:33
> 收件人: user
> 主题: HBase Filter Problem
> Hi,
> I wrote a program which function is to extract some data from a HBase
> table. According to business requirements I had to use the PrefixFilter and
> the SingleColumnValueFilter to filter the data.  The program ran very fast
> and returned in 1 sec.
> Considering I just need the rowkey of each record in my final result, I
> tried to improve my program by using the PrefixFilter +
> SingleColumnValueFilter + FirstKeyOnlyFitler. To my surprise the program
> ran very slow this time. It run about 20min and still not finished. So I
> had to kill it.
> Does anybody know the reason that cause my program run such slow?  Since I
> set the PrefixFilter as the first filter in the FilterList object, I think
> the program should ran fast.
> Many Thanks!
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message