hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod KV <vino...@yahoo-inc.com>
Subject Re: Hadoop-On-Demand (HOD) newbie: question and curious about user experience
Date Wed, 30 Apr 2008 12:21:07 GMT
Hey Alvin,

>
> First, I am wondering how the allocation actually works. For example, 
> it seems like the hod interface allows me to specify which nodes to 
> run on (in cluster_dir/hadoop-site.xml", and how many nodes to run on 
> (-o option in hod). When I execute this command, do I have to know in 
> advance that all the nodes in my cluster_dir/hadoop-site.xml are 
> available for HOD to use (i.e., HOD might feel free to wipe anything 
> on it)? And if the number of requested nodes have is smaller than the 
> hosts specified in hadoop-site.xml, how does HOD decide which nodes to 
> run on?
>

On a first note, can you please tell which documentation you are looking 
at. 'Coz hod interface is cleaned up and -o option is part of old 
interface. You can find the latest HOD documentation at 
http://hadoop.apache.org/core/docs/r0.16.3/hod.html

Hod interface lets you tell how many nodes nodes you want, and not the 
actual nodes. HOD obtains the list of nodes and thus gets its components 
(hodring and ringmaster) running by talking to a resource manager, 
torque 
(http://www.clusterresources.com/pages/products/torque-resource-manager.php) 
being the only one that hod works with currently.

> Also, HOD is advertised as: "a very useful tool for Hadoop developers 
> and testers who need to share a physical cluster for testing their own 
> Hadoop versions." Do these users need to actively coordinate which 
> nodes they use for their virtual cluster or will HOD coordinate the 
> different virtual clusters (i.e., each HOD virtual cluster is 
> essentially a batch job on the physical cluster, and uses Torque/Maui 
> to schedule)?
>

Your last point is right, each user can have his own HOD virtual 
cluster(s), with each cluster tied to a cluster directory that (s)he owns.

Please look here for a sample HOD session 
(http://hadoop.apache.org/core/docs/r0.16.3/hod_user_guide.html#A+typical+HOD+session).

HTH

-vinod



Mime
View raw message