hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "HadoopJavaVersions" by scottcarey
Date Wed, 22 Jun 2011 23:08:06 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "HadoopJavaVersions" page has been changed by scottcarey:
http://wiki.apache.org/hadoop/HadoopJavaVersions?action=diff&rev1=9&rev2=10

  If you have a problem with Hadoop, and you are running anything other than the official
Sun/Oracle JDK, you are likely to be asked to try and run Hadoop on the Oracle JDK/JVM, to
see if the problem goes away. That said, patches that help Hadoop run on other JVMs ''which
do not effect the stability or performance of Hadoop on the Oracle JVM'' are encouraged.
  
  == Sun JDK ==
- 
  Hadoop is built and tested on Oracle JDKs.  Here are the known JDKs in use or have been
tested and their status:
  
-  || '''Version''' || '''Status''' || '''Reported By''' ||
+  ||'''Version''' ||'''Status''' ||'''Reported By''' ||
-  || 1.6.0_18 || Avoid || Many ||
+  ||1.6.0_18 ||Avoid ||Many ||
-  || 1.6.0_19 || Avoid || Many ||
+  ||1.6.0_19 ||Avoid ||Many ||
-  || 1.6.0_20 || Good || LinkedIn, Cloudera ||
+  ||1.6.0_20 ||Good ||LinkedIn, Cloudera ||
-  || [[http://www.oracle.com/technetwork/java/javase/downloads/jdk6-jsp-136632.html|1.6.0_21]]
|| Good || Yahoo!, Cloudera ||
+  ||[[http://www.oracle.com/technetwork/java/javase/downloads/jdk6-jsp-136632.html|1.6.0_21]]
||Good ||Yahoo!, Cloudera ||
  
- The Sun JVM has 32-bit and 64-bit modes. In a large cluster the NameNode and JobTracker
need to run in 64-bit mode to keep all their data structures in memory.
- The workers can be set up for either 32-bit or 64-bit operation, depending upon preferences
and how much memory the individual tasks need. Problems have been reported with Hadoop, the
64-bit JVM and Compressed Object References (the -XX:+UseCompressedOops option), so use of
that option is discouraged.
  
+ 
+ The Sun JVM has 32-bit and 64-bit modes. In a large cluster the NameNode and JobTracker
need to run in 64-bit mode to keep all their data structures in memory. The workers can be
set up for either 32-bit or 64-bit operation, depending upon preferences and how much memory
the individual tasks need.
+ 
+ Using the Compressed Object References JVM feature (-XX:+UseCompressedOops) reduces memory
consumed and increases performance on 64 bit Sun JVMs.  This feature was first introduced
in 1.6.0_14 but problems have been reported with its use on versions prior to 1.6.0_20.  Several
have reported success using it on 1.6.0_21 and above.  It is the default in 1.6.0_24 and above
on 64 bit JVMs.
+ 
+ Useful tips for discovering and inspecting Sun JVM confuguration flags are in the following
blog post: [[http://q-redux.blogspot.com/2011/01/inspecting-hotspot-jvm-options.html|inspecting-hotspot-jvm-options]]
  
  == OpenJDK ==
  Hadoop does build and run on OpenJDK.
@@ -25, +28 @@

  OpenJDK is handy to have on a development system as it has more source for you to step into
when debugging something. OpenJDK and Sun JDK mainly differ in (native?) rendering/AWT/Swing
code, which is not relevant for any MapReduce Jobs that aren't creating images as part of
their work.
  
  == Oracle JRockit ==
- 
  Oracle's JRockit JVM is not the same as the Sun JVM: it has very different heap and memory
management behavior. Hadoop has been used on JRockit, though not at "production" scale.
  
   1. [[http://forums.oracle.com/forums/thread.jspa?messageID=3474253|Problems spawning jobs]]
@@ -33, +35 @@

   1. [[https://issues.apache.org/jira/browse/HADOOP-3654|Log4J configuration issues]]
  
  == IBM JDK ==
- 
- * Hadoop 0.20.2 has been tested comprehensively and works with IBM Java 6 SR 8. IBM Java
can be [[http://www.ibm.com/developerworks/java/jdk/linux/download.html | downloaded here]].

+ * Hadoop 0.20.2 has been tested comprehensively and works with IBM Java 6 SR 8. IBM Java
can be [[http://www.ibm.com/developerworks/java/jdk/linux/download.html|downloaded here]].
  
  * Hadoop 0.21.0 Common has a few Oracle specific library references in it. Patches have
been submitted for inclusion in Hadoop 0.21.1 which will resolve those errors. However, those
patches can be retrieved and applied now to make 0.21.0 work with IBM Java.
  
- * IBM also has [[http://www.alphaworks.ibm.com/tech/idah | The IBM Distribution of Apache
Hadoop]] which bundles IBM Java and a Hadoop cluster installer wizard.
+ * IBM also has [[http://www.alphaworks.ibm.com/tech/idah|The IBM Distribution of Apache
Hadoop]] which bundles IBM Java and a Hadoop cluster installer wizard.
  
  == Apache Harmony ==
- 
  [[http://harmony.apache.org/|Apache Harmony]] is the Java SE project of the Apache Software
Foundation. Within this project Harmony Select exists as a lightweight JVM with only a subset
(ex. swing, applet and others not included) of the libraries in Harmony. For the past couple
of months we have worked on the task of making all three projects in Hadoop run in Harmony
Select 6, there are still a few non critical pending issues in all three projects, but we
hope this serves as a useful guide for anyone deciding to use both Harmony and Hadoop.
  
   * [[HarmonyBuild|Building Harmony and Harmony Select 6]]
@@ -49, +49 @@

   * [[HarmonyHdfs|Hadoop hdfs on Harmony Select]]
   * [[HarmonyMapreduce|Hadoop mapreduce on Harmony Select]]
  
- 
  == A request for help from JVM/JDK developers ==
  We would strongly encourage anyone who produces a JVM/JDK to test compiling and running
Hadoop with it. It makes for a fantastic performance and stress test. As Hadoop is becoming
a key back-end datacenter application, good Hadoop support matters.
  

Mime
View raw message