hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Re: How to quickly count the rows that meet several conditions using hbase coprocessor
Date Sat, 18 Jan 2014 04:00:34 GMT
Please take a look at TestParseFilter#testCompoundFilter2

You can construct compound filter which involves more than
one QualifierFilter.

Cheers


On Fri, Jan 17, 2014 at 7:41 PM, leiwangouc@gmail.com
<leiwangouc@gmail.com>wrote:

> Hi Ted,
>
> According to the initial email,  I need two QualifierFilter and one
> SingleColumnValueFilter.  But apply 2 QualifierFilter on a scan will not
> work as described
> http://stackoverflow.com/questions/13379350/how-to-apply-several-qualifierfilter-to-a-row-in-hbaseso
i transfer them to SingleColumnValueFilter.
>
> Thanks,
> Lei
>
>
>
>
> leiwangouc@gmail.com
>
> From: Ted Yu
> Date: 2014-01-18 11:06
> To: user@hbase.apache.org
> Subject: Re: Re: How to quickly count the rows that meet several
> conditions using hbase coprocessor
> Copying the last reply from the link you gave:
>
> @Henrik I don't know how much data you have but I'm afraid you are right.
> Another option would be to implement a custom filter which takes the
> qualifier list you are looking for
>
> It was acknowledged that the proposed transformation may not give the best
> performance.
>
> From your initial email:
> bq. has column3  and the value of column3 satisfy a regular expression
>
> Why do you need 2 SingleColumnValueFilter's if you only want to check the
> value of column3 ?
>
> Cheers
>
>
> On Fri, Jan 17, 2014 at 6:47 PM, leiwangouc@gmail.com
> <leiwangouc@gmail.com>wrote:
>
> > Hi Ted,
> >
> > Haven't tried the performance without using coprocessor.
> >
> > Actually I  transfered two 2 QualifierFilter to 2 SingleColumnValueFilter
> > according to the describtion
> >
> http://stackoverflow.com/questions/13379350/how-to-apply-several-qualifierfilter-to-a-row-in-hbaseand
then pass the scan to to AggregationClient#rowCount.
> >
> > Is there anything special in regard to SingleColumnValueFilter that
> > results the slowness?
> >
> > Thanks,
> > Lei
> >
> >
> >
> >
> > leiwangouc@gmail.com
> >
> > From: Ted Yu
> > Date: 2014-01-18 09:49
> > To: user@hbase.apache.org
> > CC: user
> > Subject: Re: How to quickly count the rows that meet several conditions
> > using hbase coprocessor
> > Do you see the same slowness scanning regions with FilterList outside
> > coprocessor ?
> >
> > Thanks
> >
> > On Jan 17, 2014, at 5:24 PM, "leiwangouc@gmail.com" <
> leiwangouc@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I have tried.
> > > For a talbe with about 600 million rowkey,  just pass a single
> > QualifierFilter,  it can get the result quickly.
> > > But when i add the SingleColumnValueFilter with FilterList, it becoumes
> > very slow and i can't stand it.
> > >
> > > I think i can write my own custumed aggregation client.  Is there any
> > example or user guide about how to write custumed aggregation client
> using
> > coprocessor?
> > >
> > > Thanks,
> > > Lei
> > >
> > >
> > >
> > >
> > > leiwangouc@gmail.com
> > >
> > > From: Ted Yu
> > > Date: 2014-01-17 18:03
> > > To: user@hbase.apache.org
> > > CC: user
> > > Subject: Re: How to quickly count the rows that meet several conditions
> > using hbase coprocessor
> > > Take a look at
> >
> http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/client/coprocessor/AggregationClient.html#rowCount(byte[],%20org.apache.hadoop.hbase.coprocessor.ColumnInterpreter,%20org.apache.hadoop.hbase.client.Scan)
> > >
> > > You can pass custom filter through Scan parameter.
> > >
> > > Cheers
> > >
> > > On Jan 16, 2014, at 11:58 PM, "leiwangouc@gmail.com" <
> > leiwangouc@gmail.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> I know that hbase copocessor provides a quick way to count the rows of
> > a table.
> > >> But how can i count the rows that meet several conditions.
> > >>
> > >> Take this for example.
> > >> I have a hbase table with one column family, several columns. I want
> to
> > caculate the number of rows that meet 3 conditions:
> > >> has column1
> > >> has column2
> > >> has column3  and the value of column3 satisfy a regular expression
> > >>
> > >> Thans,
> > >> Lei
> > >>
> > >>
> > >>
> > >> leiwangouc@gmail.com
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message