hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "FAQ" by SomeOtherAccount
Date Mon, 13 Jun 2011 18:01:21 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "FAQ" page has been changed by SomeOtherAccount:

  Longer answer:  Since HFDS does not do raw disk block storage, there are two block sizes
in use when writing a file in HDFS: the HDFS blocks size and the underlying file system's
block size.  HDFS will create files up to the size of the HDFS block size as well as a meta
file that contains CRC32 checksums for that block.  The underlying file system store that
file as increments of its block size on the actual raw disk, just as it would any other file.
  = Platform Specific =
+ == General ==
+ === Problems building the C/C++ Code ===
+ While most of Hadoop is built using Java, a larger and growing portion is being rewritten
in C and C++.  As a result, the code portability between platforms is going down.  Part of
the problem is the lack of access to platforms other than Linux and our tendency to use specific
BSD, GNU, or System V functionality in places where the POSIX-usage is non-existent, difficult,
or non-performant.
+ That said, the biggest loss of native compiled code will be mostly performance of the system
and the security features present in newer releases of Hadoop.  The other Hadoop features
usually have Java analogs that work albeit slower than their C cousins.  The exception to
this is security, which absolutely requires compiled code.
  == Mac OS X ==
  === Building on Mac OS X 10.6 ===
  Be aware that Apache Hadoop 0.22 and earlier require Apache Forrest to build the documentation.
 As of Snow Leopard, Apple no longer ships Java 1.5 which Apache Forrest requires.  This can
be accomplished by either copying /System/Library/Frameworks/JavaVM.Framework/Versions/1.5
and 1.5.0 from a 10.5 machine or using a utility like Pacifist to install from an official
Apple package. http://chxor.chxo.com/post/183013153/installing-java-1-5-on-snow-leopard provides
some step-by-step directions.

View raw message