hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Java Versions and Hadoop
Date Mon, 10 Oct 2011 10:45:21 GMT
On 08/10/11 19:19, Ted Dunning wrote:
> I hate to sound like the folks who only recently stopped using 1.4, but I am
> afraid that Todd is right on here.
> The folks who are desperate for new features are being siphoned off by Scala
> and Clojure which is leaving a core of recalcitrant termagants like me.

Language wise, you are right. I actually use Groovy under Java6 for a 
lot of work these days, as it offers many of the features of the Java7 
language today yet runs on Java6, and lets you subclass java classesand 
vice versa.

Here, for example, is a MiniDFSCluster for Hadoop 0.20.20x that fixes 
its need for a JVM property to set the dest dir for data (fixed in 
trunk, BTW)


> I
> think that it is going to take something as major as the EOL of Java 6 to
> get big projects to allow code that requires Java 7.

JVM-wise, people have problems with the hotspot compiler, which means 
"don't use Hadoop with it"


I don't know if the updates fix this. Eventually they shall.

For people to be confident that it works someone needs to bring up a 
large cluster and run it on Hadoop long enough -with a more complex 
workload than terasort -to find the bugs, the race conditions, the 
problems that only surface once you serve up 2+PB of data.

I hope someone volunteers for this. As Oracle have announced a 
Hadoop-based system, they could be the people to step up here, or they 
could pay someone else to do the work.

In an ideal world, the Hadoop stack would be part of the test stack for 
a JVM release.

In the meantime, even if Oracle say Java6 is EOL, if people pay money to 
keep it alive -and they will have to in any project you don't want to 
have to requalify for java7- then it may keep going for longer, except 
the updates won't be so widely available.

I don't know if anyone with a Hadoop cluster has a support contract with 
Oracle for it. I know Yahoo! did, but I don't know it's current state.

View raw message