hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gray <jg...@facebook.com>
Subject RE: HBase as a versioned key/value store
Date Wed, 03 Nov 2010 15:28:02 GMT
Ah, reads.  Totally different story.

Are you using the block cache?  How much heap do you have configured for your RSs?

Some of that debug should be displaying the stats of your block cache... Want to paste a few
lines of that?

> -----Original Message-----
> From: Wojciech Langiewicz [mailto:wlangiewicz@gmail.com]
> Sent: Wednesday, November 03, 2010 7:15 AM
> To: user@hbase.apache.org
> Subject: Re: HBase as a versioned key/value store
> 
> Hello,
> 
> 2010/11/3 Jonathan Gray <jgray@facebook.com>
> 
> > Hi Wojciech,
> >
> > HBase can easily be used as a versioned key/value store.  I'd say
> that's
> > one of the easiest ways to use it.
> >
> > To help you get more throughput, you'll have to provide more details.
> >
> > What version are you running, what kind of hardware / configuration,
> and
> > what does your client look like that is doing the writes?
> >
> 
> I'm running latest version from Cloudera, and I'm running that script
> directly from one of the servers. All 4 of them are: 2xXeon 5400, 16GB
> RAM,
> 2x1TB HDD.
> 
> 
> > Each KV is a distinct Put operation?  Normally people get high
> throughput
> > by batching many Puts at once.
> >
> 
> Actually, here I'm asking about Get operations, because I don't know
> how to
> batch them (by design). But in case of Puts you are right.
> 
> 
> > During your writes, what do you see in the RegionServer logs?  Are
> things
> > calm or a lot of things happening?  You could also be dealing with
> some cold
> > start issues if you don't have enough regions in your table or if
> your
> > writes are not distributed across the keyspace.
> >
> 
> Writes are equally distributed, I see it with www interface,
> regionserver
> logs are calm, only debug information appears from time to time (about
> every
> minute).
> 
> I'm rather asking what can I expect from my schema design and hardware
> by
> comparing other people solutions, right now I'm getting 10 times less
> performance that I initially wanted.
> 
> 
> >
> > JG
> >
> > > -----Original Message-----
> > > From: Wojciech Langiewicz [mailto:wlangiewicz@gmail.com]
> > > Sent: Wednesday, November 03, 2010 4:19 AM
> > > To: user@hbase.apache.org
> > > Subject: HBase as a versioned key/value store
> > >
> > > Hello,
> > > I would like to know if any of is are using HBase as a versioned
> > > key/value store. What I mean by versioned is keys map to multiple
> > > values
> > > with timestamp.
> > >
> > > So the whole table would have many rows and only one column family
> with
> > > one column.
> > >
> > > I'm trying to work out what performance could I get from this
> design,
> > > because right now I can only get about 1500 - 2000 requests per
> second
> > > on 4 medium servers, and I'd like to get about 10x more. I know
> that
> > > it's not optimal use for HBase, but if only it was 10x faster I
> would't
> > > have to use any other system for this.
> > >
> > > Thanks in advance for any ideas.
> > >
> > > --
> > > Wojciech Langiewicz
> >
> 
> 
> 
> --
> Wojciech Langiewicz

Mime
View raw message