hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <ja...@attributor.com>
Subject hdfs fuse mount and namenode out of memory conditions.
Date Mon, 17 Nov 2008 17:29:20 GMT
We recently setup a fuse mount using the 18.2 fuse code, against our 
18.1 hdfs, which has been running stably for some time.

We have about 20 datanodes and 50TB or so in our hdfs. The namenode is 
running an i686 kernel and has been running with -Xmx1500m.
We have 1,492,093 files in our hdfs at present.

Since we put up the fuse mount which reads 2 large files at a time, 
about 6 a second or so, we have had the namenode OOM multiple times.

We have upped the memory foot print to the max allowed for a 32bit jvm.

What memory size should we be running?
Is the fuse mount doing something that is increasing the memory use of 
the namenode?

Happy week all.

View raw message