hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Data processing/filtering on the server
Date Wed, 13 Jan 2010 19:56:36 GMT
Coprocessors will also let you move some forms of computation into the
region server process as that makes sense. See HBASE-2000.

    http://issues.apache.org/jira/browse/HBASE-2000

This issue *is* under active development. I've not had much time as of 
late but when things calm down at work I'll be back at it. 

For example, with coprocessors it would be possible to run MapReduce
computations on data stored in HBase without all of the data transfer
and task scheduling overheads of involving the Hadoop MapReduce
framework. Speed ups of 100x or 1000x are not unreasonable to 
contemplate. 

   - Andy



----- Original Message ----
> From: Sriram Muthuswamy Chittathoor <sriramc@ivycomptech.com>
> To: hbase-user@hadoop.apache.org
> Sent: Wed, January 13, 2010 9:38:44 AM
> Subject: RE: Data processing/filtering on the server
> 
> Thanks Edward.  As an experiment I was trying to write my own filter and use the 
> scan.setFilter (..)  method to set it.  Is this even possible as I got into 
> issues --  started giving some region server exception.  Based on your response 
> if the filtering is applied on the server side obviously my local custom filter 
> class cannot be used.  Am I guessing it right ??
> 
> -----Original Message-----
> From: Edward Capriolo [mailto:edlinuxguru@gmail.com] 
> Sent: Wednesday, January 13, 2010 11:00 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Data processing/filtering on the server
> 
> Filters are applied server side:
> 
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/filter/package-summary.html
> 
> Edward
> On Wed, Jan 13, 2010 at 12:28 PM, Sriram Muthuswamy Chittathoor
> wrote:
> > Hi:
> >
> > I am currently using the HBase client api to fetch the data rows and
> > process the rows on the client JVM.  Is there a way to specify filters
> > which will apply say on the region servers so that the data that is
> > returned to me is a filtered set.  I don't want all the data rows to
> > travel on the wire from the server to the client JVM
> >
> > Sriram C
> >
> > This email is sent for and on behalf of Ivy Comptech Private Limited. Ivy 
> Comptech Private Limited is a limited liability company.
> >
> > This email and any attachments are confidential, and may be legally privileged 
> and protected by copyright. If you are not the intended recipient dissemination 
> or copying of this email is prohibited. If you have received this in error, 
> please notify the sender by replying by email and then delete the email 
> completely from your system.
> > Any views or opinions are solely those of the sender.  This communication is 
> not intended to form a binding contract on behalf of Ivy Comptech Private 
> Limited unless expressly indicated to the contrary and properly authorised. Any 
> actions taken on the basis of this email are at the recipient's own risk.
> >
> > Registered office:
> > Ivy Comptech Private Limited, Cyber Spazio, Road No. 2, Banjara Hills, 
> Hyderabad 500 033, Andhra Pradesh, India. Registered number: 37994. Registered 
> in India. A list of members' names is available for inspection at the registered 
> office.
> >
> >
> 
> This email is sent for and on behalf of Ivy Comptech Private Limited. Ivy 
> Comptech Private Limited is a limited liability company.  
> 
> This email and any attachments are confidential, and may be legally privileged 
> and protected by copyright. If you are not the intended recipient dissemination 
> or copying of this email is prohibited. If you have received this in error, 
> please notify the sender by replying by email and then delete the email 
> completely from your system. 
> Any views or opinions are solely those of the sender.  This communication is not 
> intended to form a binding contract on behalf of Ivy Comptech Private Limited 
> unless expressly indicated to the contrary and properly authorised. Any actions 
> taken on the basis of this email are at the recipient's own risk.
> 
> Registered office:
> Ivy Comptech Private Limited, Cyber Spazio, Road No. 2, Banjara Hills, Hyderabad 
> 500 033, Andhra Pradesh, India. Registered number: 37994. Registered in India. A 
> list of members' names is available for inspection at the registered office.



      


Mime
View raw message