hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikram Singh Chandel <vikramsinghchan...@gmail.com>
Subject Re: Why Custom Filters are more resource exhaustive then normal single column value filter
Date Tue, 17 Jun 2014 13:55:17 GMT
Hi Ramkrishna

issues with CoPro there are many
1: Garbage Collection
2: JVM Freeze
3: Removal of CoPro from RS unless you do a rolling restart
4: Class unloading
5:Class Collision Issues
6:Shared JVM between CoPro and RS

there may be some more that i am not aware of :-)


On Tue, Jun 17, 2014 at 5:38 PM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> >> If they don't
> process records parallel on RS why not use HDFS to distribute the jar?
> See https://issues.apache.org/jira/browse/HBASE-1936 for dynamic class
> loading of jars.
>
> In the first mail you had mentioned that there are various problems in CP?
> What are those problems?  Filters are mainly used to work on the Rows/cells
> that are retrieved in every region.  Whereas CPs are used to add some user
> extensions to the way Servers works in write path or read path.
>
> Not sure if I understand your problem correctly.
>
>
> On Tue, Jun 17, 2014 at 5:20 PM, Vikram Singh Chandel <
> vikramsinghchandel@gmail.com> wrote:
>
> > Hi
> > ya Anoop the comparison was between SCVF and Custom Filter. Seeing the
> > deployment process of Custom Filter, i.e building a jar deploying it on
> to
> > all RS
> > *i thought that the work in similar fashion as that of CP*. If they don't
> > process records parallel on RS why not use HDFS to distribute the jar?
> >
> >
> > On Mon, Jun 16, 2014 at 6:52 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > Can you format the table for better reading ?
> > > You can put picture on third party website and post link here.
> > >
> > > Does your custom filter maintain state ?
> > >
> > > What hbase version are you using ?
> > >
> > > Have you checked region server logs during the 15 minutes ?
> > >
> > > Cheers
> > >
> > > On Jun 16, 2014, at 3:25 AM, Vikram Singh Chandel <
> > > vikramsinghchandel@gmail.com> wrote:
> > >
> > > > Hi
> > > > I was working on way to find a substitute for coprocessor(because of
> > > > various issue with them)
> > > > so tried Custom Filters and found that the custom filter are much
> more
> > > > resource exhaustive (CPU) then coprocessor having single column value
> > > > filter.
> > > >
> > > > *Query: Get List of Publications for UDANDA (CF:Attributes
> > Col:Country) -
> > > > Result: 509 Records. *
> > > >
> > > > *Heap 3 Gb *
> > > >
> > > >
> > > > With Custom Filter
> > > > No of Scan Req
> > > > Max Heap(Mb)
> > > > Max CPU(%)
> > > >
> > > >
> > > >
> > > > 1
> > > > 86.74
> > > > 7.7
> > > > 5
> > > > 412
> > > > 63.37
> > > > 10
> > > > 926
> > > > 99
> > > > *With Single Column Value Filter*
> > > > No of Scan Req
> > > > Max Heap
> > > > Max CPU
> > > >
> > > >
> > > >
> > > > 10
> > > > 453.66
> > > > 17.7
> > > > Query ran for 15 minutes didn't get results. Terminated execution
> > > >
> > > >
> > > >
> > > > *Can anyone shed some light on this.*
> > > > --
> > > >
> > > > *Regards*
> > > >
> > > > *VIKRAM SINGH CHANDEL*
> > > >
> > > > Please do not print this email unless it is absolutely
> > necessary,Reduce.
> > > > Reuse. Recycle. Save our planet.
> > >
> >
> >
> >
> > --
> > *Regards*
> >
> > *VIKRAM SINGH CHANDEL*
> >
> > Please do not print this email unless it is absolutely necessary,Reduce.
> > Reuse. Recycle. Save our planet.
> >
>



-- 
*Regards*

*VIKRAM SINGH CHANDEL*

Please do not print this email unless it is absolutely necessary,Reduce.
Reuse. Recycle. Save our planet.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message