hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M. C. Srivas" <mcsri...@gmail.com>
Subject Re: Best Hbase Storage for PIG
Date Sat, 28 Apr 2012 15:16:39 GMT
On Thu, Apr 26, 2012 at 4:38 AM, Rajgopal Vaithiyanathan <
raja.fire@gmail.com> wrote:

> Hey all,
>
> The default - HBaseStorage() takes hell lot of time for puts.
>
> In a cluster of 5 machines, insertion of 175 Million records took 4Hours 45
> minutes
> Question -  Is this good enough ?
> each machine has 32 cores and 32GB ram with 7*600GB harddisks. HBASE's heap
> has been configured to 8GB.
> If the put speed is low, how can i improve them..?
>

Raj, how big is each record?



>
> I tried tweaking the TableOutputFormat by increasing the WriteBufferSize to
> 24MB, and adding the multi put feature (by adding 10,000 puts in ArrayList
> and putting it as a batch).  After doing this,  it started throwing
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message