hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "HdfsFutures" by SanjayRadia
Date Fri, 13 Feb 2009 22:16:24 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by SanjayRadia:
http://wiki.apache.org/hadoop/HdfsFutures

------------------------------------------------------------------------------
  
  
  === Scaling Namespace (i.e. number of files/dirs) ===
+ Since the name node stores block and name objects in memory, the size of the name space
(and hence the number of files) is limited by amount of heap memory. Currently a 14GB heap
(ie 16GB machine) allows 60 million block and name objects. Hence
+ if one has 2 blocks per file, then one is limited to 20 million files.
  
  Partition/distribute Name node (will also help performance)
     [[BR]]Several Options:

Mime
View raw message