hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <...@yahoo-inc.com>
Subject Re: Hadoop + WinXP + cygwin
Date Tue, 23 May 2006 23:11:43 GMT
Krzysztof KucybaƂa wrote:

> Hi!
> I am new to hadoop as well as cygwin, and as far as I know, you need 
> to use cygwin in order to use hadoop on win. Unfortunately I'm not 
> allowed to switch to linux or even use a linux machine to get the dfs 
> running. Seems there's no other way but the cygwin-way, is there? 

If you use dfs only then you can replace one class DF.java with a 
universal version see attachment in
and run the cluster without cygwin. I do.
If you are planning to use map/reduce then cygwin is probably the best 
way, since you want to start
job/task Tracker using hadoop scripts.

> Anyways, I was wondering, is there a way to get hadoop daemons running 
> via cygwin, and then quit the latter? Cause I think I got the namenode 
> and datanode running (how can I test that, by the way - other than by 
> writing "ps" in cygwin?

Use bin/hadoop dfs -ls /
or other options. This is a command line shell. Under cygwin.

> Does writing the address and port of the node in a browser and waiting 
> for the outcome, which is a blank-but-not-error page, tell me anything 
> about whether the dfs is configured correctly?), yet if I close 
> cygwin, the daemons shutdown too.
> And there's another thing I'd like to ask. I'm writing a Java program 
> that is supposed to connect to a dfs. As much as I've read the API 
> docs, I suppose I should use the DistributedFileSystem class, 
> shouldn't I? 

You should use FileSystem class see
and test/org.apache.hadoop.fs

> But what does creating it's instance actually do? Create me a new 
> filesystem or rather just a connection to an existing one? What I do 
> is specify a InetSocketAddress and a Configuration. 

FileSystem.get( Configuration ) would do

> Can the configuration object be created using the hadoop-defaul.xml 
> and hadoop-site.xml files?

This is the default behavior. Configuration constructor reads the files.

> I know these questions probably sound stupid, but still I'd really 
> appreciate if someone provided me with some answers. I'm a true 
> beginner  in the matters of hadoop and cygwin, and I'm also quite new 
> to Java, so please - be gentle ;o)
> Regards,

View raw message