hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Virtual Hadoop" by SteveLoughran
Date Wed, 05 Oct 2011 20:53:43 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Virtual Hadoop" page has been changed by SteveLoughran:
http://wiki.apache.org/hadoop/Virtual%20Hadoop?action=diff&rev1=2&rev2=3

Comment:
split infra from hadoop aspects of datacentre reboot

   1. High performance infrastructure services aid Hadoop (DNS, reverse DNS, NFS storage for
NameNode snapshots)
   1. The primary failure modes of machines are HDD failures, re-occurring memory failures,
or overheating damage caused by fan failures.
   1. Machine failures are normally independent, with the exception of the failure of Top
of Rack switches, which can take a whole rack offline. Router/Switch misconfigurations can
have a similar effect.
-  1. If the entire datacenter restarts, the filesystem can recover, provided you have set
up the NameNode and Secondary NameNode properly.
+  1. If the entire datacenter restarts, almost all the machines will come back up -along
with their data.
  
  === Hadoop's implementation details ===
  
@@ -49, +49 @@

   1. Servers that are consistently slow to complete jobs should be blacklisted: no new work
should be sent to them. 
   1. The JobTracker should try and keep the cluster as busy as possible, to maximise ROI
on the servers and datacenter.
   1. When a JobTracker has no work to perform, the servers are left idle. 
+  1. If the entire datacenter restarts, the filesystem can recover, provided you have set
up the NameNode and Secondary NameNode properly.
  
  === How a virtual infrastructure differs from a physical datacenter ===
  

Mime
View raw message