hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: Running Hadoop on cluster with NFS booted systems
Date Thu, 24 Sep 2009 13:03:31 GMT
Hey Nick,

Never seen that behavior.  Couldn't imagine why it'd be an issue,  
unless there was something different between your local OS install and  
netboot.

Have you tried looking into the logs?  They'd state pretty explicitly  
that the datanode isn't connecting to the namenode.

Another thing to consider -- hostname lookup and reverse-lookup ought  
to be working at the time HDFS is started; it's quite picky about  
this.  Maybe you need to add HDFS later on to the boot sequence?

Brian

On Sep 23, 2009, at 4:42 PM, Nick Rathke wrote:

> Hi,
>
> I am hopping that someone can help with this issue. I have a 64 node  
> cluster that we would like to run Hadoop on, most of the nodes are  
> netbooted via NFS.
>
> Hadoop runs fine on nodes IF the node uses a local OS install, but  
> don't work when I netboot. Under netboot I can see that the slaves  
> have the correct Java processes running, but the hadoop web page  
> never shows the nodes as available.
>
> On the few node that have a local OS installed everything works just  
> fine and I can run the test jobs without issue (so far).
>
> Has anyone had this issue ?
>
> TIA
>
> -- 
> Nick Rathke
> Scientific Computing and Imaging Institute
> Sr. Systems Administrator
> nick@sci.utah.edu
> www.sci.utah.edu
> 801-587-9933
> 801-557-3832
>
> "I came I saw I made it possible" Royal Bliss - Here They Come


Mime
View raw message