hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Re: How to quickly count the rows that meet several conditions using hbase coprocessor
Date Sat, 18 Jan 2014 03:06:39 GMT
Copying the last reply from the link you gave:

@Henrik I don't know how much data you have but I'm afraid you are right.
Another option would be to implement a custom filter which takes the
qualifier list you are looking for

It was acknowledged that the proposed transformation may not give the best
performance.

>From your initial email:
bq. has column3  and the value of column3 satisfy a regular expression

Why do you need 2 SingleColumnValueFilter's if you only want to check the
value of column3 ?

Cheers


On Fri, Jan 17, 2014 at 6:47 PM, leiwangouc@gmail.com
<leiwangouc@gmail.com>wrote:

> Hi Ted,
>
> Haven't tried the performance without using coprocessor.
>
> Actually I  transfered two 2 QualifierFilter to 2 SingleColumnValueFilter
> according to the describtion
> http://stackoverflow.com/questions/13379350/how-to-apply-several-qualifierfilter-to-a-row-in-hbase
and then pass the scan to to AggregationClient#rowCount.
>
> Is there anything special in regard to SingleColumnValueFilter that
> results the slowness?
>
> Thanks,
> Lei
>
>
>
>
> leiwangouc@gmail.com
>
> From: Ted Yu
> Date: 2014-01-18 09:49
> To: user@hbase.apache.org
> CC: user
> Subject: Re: How to quickly count the rows that meet several conditions
> using hbase coprocessor
> Do you see the same slowness scanning regions with FilterList outside
> coprocessor ?
>
> Thanks
>
> On Jan 17, 2014, at 5:24 PM, "leiwangouc@gmail.com" <leiwangouc@gmail.com>
> wrote:
>
> > Hi,
> >
> > I have tried.
> > For a talbe with about 600 million rowkey,  just pass a single
> QualifierFilter,  it can get the result quickly.
> > But when i add the SingleColumnValueFilter with FilterList, it becoumes
> very slow and i can't stand it.
> >
> > I think i can write my own custumed aggregation client.  Is there any
> example or user guide about how to write custumed aggregation client using
> coprocessor?
> >
> > Thanks,
> > Lei
> >
> >
> >
> >
> > leiwangouc@gmail.com
> >
> > From: Ted Yu
> > Date: 2014-01-17 18:03
> > To: user@hbase.apache.org
> > CC: user
> > Subject: Re: How to quickly count the rows that meet several conditions
> using hbase coprocessor
> > Take a look at
> http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/client/coprocessor/AggregationClient.html#rowCount(byte[],%20org.apache.hadoop.hbase.coprocessor.ColumnInterpreter,%20org.apache.hadoop.hbase.client.Scan)
> >
> > You can pass custom filter through Scan parameter.
> >
> > Cheers
> >
> > On Jan 16, 2014, at 11:58 PM, "leiwangouc@gmail.com" <
> leiwangouc@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> I know that hbase copocessor provides a quick way to count the rows of
> a table.
> >> But how can i count the rows that meet several conditions.
> >>
> >> Take this for example.
> >> I have a hbase table with one column family, several columns. I want to
> caculate the number of rows that meet 3 conditions:
> >> has column1
> >> has column2
> >> has column3  and the value of column3 satisfy a regular expression
> >>
> >> Thans,
> >> Lei
> >>
> >>
> >>
> >> leiwangouc@gmail.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message