hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy" <sa...@pearsonwholesale.com>
Subject hbase master heap space
Date Thu, 20 Dec 2007 20:37:08 GMT
I not sure of this but why does the master server use up so much memory. I 
been running an script that been inserting data into a table for a little 
over 24 hours and the master crashed because of java.lang.OutOfMemoryError: 
Java heap space.

So my question is why does the master use up so much memory at most it 
should store the -ROOT-,.META. tables in memory and block to table mapping.

Is it cache or a memory leak?

I am using the rest interface so could that be the reason?

I inserted according to the high edit ids on all the region servers about
51,932,760 edits and the master ran out of memory with a heap of about 1GB.

The other side to this is the data I inserted is only taking up 886.61 MB 
and that's with
dfs.replication set to 2 so half that is only 440MB of data compressed at 
the block level.
>From what I understand the master should have lower memory and cpu usage and 
the namenode on hadoop should be the memory hog it has to keep up with all 
the data about the blocks. 

View raw message