hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurent Hatier <laurent.hat...@gmail.com>
Subject Re: Insert a lot of data in HBase
Date Mon, 20 Jun 2011 19:14:43 GMT
my keys are the moment where the data is inserted into HBase (so
System.currentTimeMillis()*1000). As you can see, i use the put method which
insert data... there is an another way to insert data ?

2011/6/20 Doug Meil <doug.meil@explorysmedical.com>

>
> Look here in the HBase book for these, and other, tips.
>
> http://hbase.apache.org/book.html#performance
>
>
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of
> Jean-Daniel Cryans
> Sent: Monday, June 20, 2011 2:03 PM
> To: user@hbase.apache.org
> Subject: Re: Insert a lot of data in HBase
>
> 4M is small data :)
>
> Could there be an overlap in the keys? Are you disabling autoflush and not
> flushing the write buffer? These are common errors/misconceptions.
>
> J-D
>
> On Mon, Jun 20, 2011 at 10:36 AM, Laurent Hatier <laurent.hatier@gmail.com>
> wrote:
> > Hi all,
> >
> > I'm new in HBase. I want to insert 4'000'000 rows in HBase (each row
> > has 4 columns). I have already looked the HBase wiki to insert data,
> > but i've a problem : i loss data. When i do a COUNT with the shell,
> > there is approximativly 1'500'000 in the DB...
> > I've tested to create multiple Put and insert it with a List, i've
> > already tested a simple Put with four add functions, open and close
> > the socket it each time i put the line or i read the file don't run...
> > If anyone have an idea.
> >
> > here we go my code if you want to see :
> >
> > List<Put> arrayPut = new ArrayList<Put>();
> >
> > arrayPut.add(new Put(Bytes.toBytes(id)));
> > arrayPut.get(arrayPut.size() - 1).add(FAMILY_GEOLOC, QUALIFIER_START,
> > Bytes.toBytes(tStart)); arrayPut.add(new Put(Bytes.toBytes(id)));
> > arrayPut.get(arrayPut.size() - 1).add(FAMILY_GEOLOC, QUALIFIER_END,
> > Bytes.toBytes(tEnd)); arrayPut.add(new Put(Bytes.toBytes(id)));
> > arrayPut.get(arrayPut.size() - 1).add(FAMILY_GEOLOC,
> > QUALIFIER_COUNTRY, Bytes.toBytes(countryCode)); arrayPut.add(new
> > Put(Bytes.toBytes(id)));
> > arrayPut.get(arrayPut.size() - 1).add(FAMILY_GEOLOC, QUALIFIER_REGION,
> > Bytes.toBytes(regionCode)); table.put(arrayPut);
> >
> > --
> > Laurent HATIER
> > Étudiant en 2e année du Cycle Ingénieur à l'EISTI
> >
>



-- 
Laurent HATIER
Étudiant en 2e année du Cycle Ingénieur à l'EISTI

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message