hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "FrontPage" by OwenOMalley
Date Tue, 08 Aug 2006 21:29:31 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by OwenOMalley:
http://wiki.apache.org/lucene-hadoop/FrontPage

------------------------------------------------------------------------------
  [http://lucene.apache.org/hadoop/ Hadoop] is a framework for running applications on large
clusters built of commodity hardware. The Hadoop framework transparently provides applications
both reliability and data motion. Hadoop implements a computational paradigm named [:HadoopMapReduce:
Map/Reduce], where the application is divided into many small fragments of work, each of which
may be executed or reexecuted on any node in the cluster. In addition, it provides a distributed
file system that stores data on the compute nodes and thereby provides very high aggregate
bandwidth across the cluster. Both Map/Reduce and the distributed file system are designed
so that node failures are automatically handled by the network.
  
  The intent is to scale Hadoop up to handling thousand of computers. The current high water
marks that have been reported are:
-  * !DataNodes: 620
-  * !TaskTrackers: 500
+  * Nodes in a single file system cluster (!DataNodes): 620
+  * Nodes in a single map/reduce cluster (!TaskTrackers): 500
  
  Hadoop was originally built as infrastructure for the [http://lucene.apache.org/nutch/ Nutch]
project, which crawls the web and builds a search engine index for the crawled pages. Both
Hadoop and Nutch are part of the [http://lucene.apache.org/java/docs/index.html Lucene] [http://www.apache.org/
Apache] project.
  

Mime
View raw message