hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject RE: Inserting Random Data into HBASE
Date Wed, 01 Dec 2010 21:02:06 GMT


Ok, this may be a silly question ... 

Why aren't you using the cache on the client. When you have client side cache, your actual
writes from a put() aren't written until you explicitly flush the buffered writes, or you
fill the the buffer/cache.

HTH

-Mike

> Date: Wed, 1 Dec 2010 18:11:40 +0200
> Subject: Re: Inserting Random Data into HBASE
> From: alex.baranov.v@gmail.com
> To: user@hbase.apache.org
> 
> Well, if you are talking about importing large amount of (existing) data you
> should look at bulk loads (
> http://hbase.apache.org/docs/r0.89.20100924/bulk-loads.html).
> 
> In case you still want to measure write performance using Puts, then make
> sure you use proper writeBufferSize and (if it is ok in your situation)
> writeToWAL settings.
> 
> In general, this question (and related) raised a lot of times on mailing
> list, try to search for relevant info (e.g. using search-hadoop.com service)
> first and ask more specific questions (and please include more info about
> your hardware, etc.).
> 
> Hope this helps,
> 
> Alex Baranau
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase
> 
> On Wed, Dec 1, 2010 at 4:47 PM, rajgopalv <raja.fire@gmail.com> wrote:
> 
> >
> > Hi,
> > I have to test hbase as to how long it takes to store 100 Million Records.
> >
> > So i wrote a simple java code which
> >
> > 1 : generates random key and 10 columns per key and random values for the
> > 10
> > columns.
> > 2 : I make a Put object out of these and store it in arrayList
> > 3 : When arrayList's size reaches 5000 i do table.put(listOfPuts);
> > 4 : repeat until i put 100 million records.
> >
> > And i run this java program as single threaded java program.
> >
> > Am i doing it right? is there any other way of importing large data for
> > testing.? [ for now i'm not considering BULK data import/loadtable.rb etc.
> > apart from this is there any other way ?]
> >
> >
> > --
> > View this message in context:
> > http://old.nabble.com/Inserting-Random-Data-into-HBASE-tp30349594p30349594.html
> > Sent from the HBase User mailing list archive at Nabble.com.
> >
> >
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message