hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gray <jg...@fb.com>
Subject RE: question about multi-transaction queries
Date Fri, 17 Dec 2010 22:26:26 GMT
Have you looked at the thrift support?  Plenty of people are using HBase from PHP via Thrift.

I don't think there is MultiPut or MultiGet support but there is work currently underway updating
the thrift API.  I imagine those two could be added.

> -----Original Message-----
> From: Jack Levin [mailto:magnito@gmail.com]
> Sent: Friday, December 17, 2010 1:45 PM
> To: user@hbase.apache.org
> Cc: user@hbase.apache.org
> Subject: Re: question about multi-transaction queries
> 
> We will have php querying hbase over tcp, and we need a connector on the
> hbase end to return content the fastest way possible
> 
> -Jack
> 
> 
> On Dec 17, 2010, at 1:32 PM, Jonathan Gray <jgray@fb.com> wrote:
> 
> > I'm not sure I understand.
> >
> > Are you trying to build a client?  Or you want something that behaves like
> the mysql client?
> >
> >> -----Original Message-----
> >> From: Jack Levin [mailto:magnito@gmail.com]
> >> Sent: Friday, December 17, 2010 1:28 PM
> >> To: user@hbase.apache.org
> >> Cc: user@hbase.apache.org
> >> Subject: Re: question about multi-transaction queries
> >>
> >> Client is a tcp framework similar to mysql client that should be able
> >> to send
> >> 1000 gets in one transaction, like a json obj that has all the keys
> >>
> >> -Jack
> >>
> >>
> >> On Dec 17, 2010, at 1:21 PM, Jonathan Gray <jgray@fb.com> wrote:
> >>
> >>> I'm not sure exactly what your requirements are but what exactly is
> >>> your
> >> client interface?  There is no persistent process anywhere serving
> >> client requests?
> >>>
> >>>> -----Original Message-----
> >>>> From: Jack Levin [mailto:magnito@gmail.com]
> >>>> Sent: Friday, December 17, 2010 12:44 PM
> >>>> To: user@hbase.apache.org
> >>>> Subject: Re: question about multi-transaction queries
> >>>>
> >>>> Do you happen to know if anyone have written or using something
> >>>> like that as open source? I would imagine this being super useful.
> >>>> There is a question of interface too, I assume it would be TCP.  Is
> >>>> there sort of Jetty plugin available?  Now I somewhat realize that
> >>>> I am just describing existing REST, but afaik, it does not support multi-
> get.
> >>>>
> >>>> -Jack
> >>>>
> >>>> On Fri, Dec 17, 2010 at 11:57 AM, Jonathan Gray <jgray@fb.com>
> wrote:
> >>>>> Yes, some kind of running JVM.  I would not recommend starting a
> >>>>> JVM for each query :)
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Jack Levin [mailto:magnito@gmail.com]
> >>>>>> Sent: Friday, December 17, 2010 11:28 AM
> >>>>>> To: user@hbase.apache.org
> >>>>>> Subject: Re: question about multi-transaction queries
> >>>>>>
> >>>>>> Ok, does it mean though we would incur Java startup cost?  Or
do
> >>>>>> you propose we write some sort of java server that has the JVM
> >>>>>> running and is able to get multi-get queries?
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> -Jack
> >>>>>>
> >>>>>> On Fri, Dec 17, 2010 at 11:15 AM, Jonathan Gray <jgray@fb.com>
> >> wrote:
> >>>>>>> All of my experience doing something like this was with
straight
> Java.
> >>>>>>>
> >>>>>>> There are MultiGet and MultiPut capabilities in the Java
client
> >>>>>>> that will help
> >>>>>> you out significantly.
> >>>>>>>
> >>>>>>> I played with Jython and HBase a couple years ago and back
then
> >>>>>>> the
> >>>>>> performance was horrible.  I never looked back but I have no
idea
> >>>>>> if it's gotten better in the meantime.
> >>>>>>>
> >>>>>>> JG
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Jack Levin [mailto:magnito@gmail.com]
> >>>>>>>> Sent: Friday, December 17, 2010 11:01 AM
> >>>>>>>> To: user@hbase.apache.org
> >>>>>>>> Subject: Re: question about multi-transaction queries
> >>>>>>>>
> >>>>>>>> Lets just say its one row key with two columns.  Non
contiguous
> >>>>>>>> records.  We want to read as fast as possible.  So we
did some
> >>>>>>>> tests, and with MongoDB the random reads of 1000 records
is
> >>>>>>>> about
> >>>> 80ms.
> >>>>>>>> While HBASE with jython is 400ms or so.
> >>>>>>>> Question is, as we develop our applications what is
the best
> >>>>>>>> method to retrieve many rows the fastest way possible?
 We are
> >>>>>>>> talking about
> >>>>>>>> 1 client here, not many clients.  For many clients,
REST seems
> >>>>>>>> to be appropriate, but here we have a Frontend server
rendering
> >>>>>>>> content quickly and we need to reduce the query overhead
for
> >>>>>>>> HBASE and get
> >>>>>> data fast.
> >>>>>>>>
> >>>>>>>> -Jack
> >>>>>>>>
> >>>>>>>> On Sat, Dec 11, 2010 at 10:55 AM, Stack <stack@duboce.net>
> wrote:
> >>>>>>>>> How many columns?  Its columns right, and not column
families?
> >>>>>>>>>
> >>>>>>>>> Are the 1k rows contiguous?  Can you Scan?  For
insert of 1k
> >>>>>>>>> rows, you know how to do that now, right?  Will
they be
> >>>>>>>>> substantial rows
> >>>>>>>>> -- 10s to 100s of ks? -- or just small?  Do you
have multiput
> >>>>>>>>> available in the REST interface, I don't recall.
> >>>>>>>>>
> >>>>>>>>> Try REST since you know that interface.  Jython
might be
> >>>>>>>>> faster though a test done more than a year ago had
jython as
> >>>>>>>>> slow
> >>>>>>>>> (http://ryantwopointoh.blogspot.com/2009/01/performance-
> of-
> >>>> hbase
> >>>>>>>>> -
> >>>>>>>> impor
> >>>>>>>>> ting.html) but a bunch has changed since then --
hbase-wise
> >>>>>>>>> and jython has probably gotten a lot better.  If
jython route,
> >>>>>>>>> make sure you keep the interpreter afloat rather
than launch
> >>>>>>>>> it per request (so yes, fastcgi would make sense).
> >>>>>>>>>
> >>>>>>>>> St.Ack
> >>>>>>>>>
> >>>>>>>>> On Fri, Dec 10, 2010 at 9:59 PM, Jack Levin
> >>>>>>>>> <magnito@gmail.com>
> >>>> wrote:
> >>>>>>>>>> Hello.   We plan to run a set of queries on
tables with
> >>>>>>>>>> multiple columns.  What is the most efficient
method to say,
> >>>>>>>>>> insert 1000 rows, and/or read 1000 rows.
> >>>>>>>>>> We are considering just using REST.  But what
about jython?
> >>>>>>>>>> Will it be faster?  Another way to have our
apps talk to
> >>>>>>>>>> nginx and some sort of app tier running via
fast-cgi.
> >>>>>>>>>>
> >>>>>>>>>> Any ideas?
> >>>>>>>>>>
> >>>>>>>>>> -Jack
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>

Mime
View raw message