hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkatesh <vramanatha...@aol.com>
Subject Re: HBase map reduce job timing
Date Wed, 06 Oct 2010 15:07:50 GMT

 Also, do you think if I query using rowkey instead of hbase time stamp..it would not kick
off that many tasks..
since region server knows the exact locations?

thanks
venkatesh

 


 

 

-----Original Message-----
From: Venkatesh <vramanathan00@aol.com>
To: user@hbase.apache.org
Sent: Wed, Oct 6, 2010 8:53 am
Subject: Re: HBase map reduce job timing


 Ahh ..ok..That makes sense

I've a 10 node cluster each with 36 gig..I've allocated 4gig for HBase Region Servers..master.jsp
reports used heap is less than half on each region server.


 I've close to 800 regions total..Guess it needs to kick off a jvm to see if data exists
in all regions..


 

 

-----Original Message-----
From: Jean-Daniel Cryans <jdcryans@apache.org>
To: user@hbase.apache.org
Sent: Tue, Oct 5, 2010 11:52 pm
Subject: Re: HBase map reduce job timing


> Regarding number of map tasks 500+, 490 of them processing nothing, do you 

have an explanation

> for that?..Wondering if its kicking off too many JVMs most doing nothing..



This would mean that throughout your regions, only a few have data in

the timestamp range you're looking for.



>

> 'top' reports less free memory (couple of gig.) though box has 36 gig total.. 

I don't quite trust

> top since cached blocks don't show up under free column even if no process is 

running..

>



You only have 1 machine?



BTW how much RAM did you give to HBase?



J-D


 
=
 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message