hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Habermaas" <b...@habermaas.us>
Subject RE: Hadoop on Windows
Date Thu, 17 Sep 2009 17:02:37 GMT
It's interesting that Hadoop, being written entirely in Java, has such a
spotty reputation running on different platforms. I had to patch it to run
on AIX and need cygwin (gack!) so it will run on Windows. I'm surprised
nobody has thought about removing it's use of bash to run system commands
(which is NOT especially portable). Now that Hadoop only comes only in a
Java 1.6 flavor why can't it figure out disk space using the native java
runtime instead of executing the DF command under bash? Of course it runs
other system commands as well which in my opinion isn't too cool. 


-----Original Message-----
From: Steve Loughran [mailto:stevel@apache.org] 
Sent: Thursday, September 17, 2009 12:53 PM
To: common-user@hadoop.apache.org
Subject: Re: Hadoop on Windows

brien colwell wrote:
> Our cygwin/windows nodes are picky about the machines they work on. On
> some they are unreliable. On some they work perfectly.
> We've had two main issues with cygwin nodes.
> Hadoop resolves paths in strange ways, so for example /dir is
> interpreted as c:/dir not %cygwin_home%/dir. For SSH to a cygwin node,
> /dir is interpreted as %cygwin_home%/dir. So our maintenance scripts
> have to make a distinction between cygwin and linux to adjust for
> Hadoop's path behavior.

That's exactly the same as any Java File instance would work on windows, 
new File("/dir") would map to c:/dir.

As the Ant team say in their docs
"We get lots of support calls from Cygwin users. Either it is incredibly 
popular, or it is trouble. If you do use it, remember that Java is a 
Windows application, so Ant is running in a Windows process, not a 
Cygwin one. This will save us having to mark your bug reports as invalid. "

View raw message