hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "HdfsFutures" by SanjayRadia
Date Fri, 13 Feb 2009 22:46:15 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by SanjayRadia:

  === Scaling Namespace (i.e. number of files/dirs) ===
  Since the name node stores block and name objects in memory, the size of the name space
(and hence the number of files) is limited by amount of heap memory. Currently a 14GB heap
(ie 16GB machine) allows 60 million block and name objects. Hence
  if one has 2 blocks per file, then one is limited to 20 million files.
+ This is a significant restriction for large clusters. Besides adding more memory, several
options are listed below.
  Partition/distribute Name node (will also help performance)
     [[BR]]Several Options:

View raw message