hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "FAQ" by SomeOtherAccount
Date Mon, 04 Apr 2011 16:56:08 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "FAQ" page has been changed by SomeOtherAccount.
http://wiki.apache.org/hadoop/FAQ?action=diff&rev1=93&rev2=94

--------------------------------------------------

   * [[http://svn.apache.org/viewvc/hadoop/core/trunk/src/c++/libhdfs|libhdfs]], a JNI-based
C API for talking to hdfs (only).
   * [[http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/pipes/package-summary.html|Hadoop
Pipes]], a [[http://www.swig.org/|SWIG]]-compatible  C++ API (non-JNI) to write map-reduce
jobs.
  
- == What is the Distributed Cache used for? ==
+ == How do I submit extra content (jars, static files, etc) for my job to use during runtime?
==
- The distributed cache is used to distribute large read-only files that are needed by map/reduce
jobs to the cluster. The framework will copy the necessary files from a url (either hdfs:
or http:) on to the slave node before any tasks for the job are executed on that node. The
files are only copied once per job and so should not be modified by the application.
+ The [[http://hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/filecache/DistributedCache.html|distributed
cache]] is used to distribute large read-only files that are needed by map/reduce jobs to
the cluster. The framework will copy the necessary files from a url (either hdfs: or http:)
on to the slave node before any tasks for the job are executed on that node. The files are
only copied once per job and so should not be modified by the application.
  
  == How do I get my MapReduce Java Program to read the Cluster's set configuration and not
just defaults? ==
  The configuration property files ({core|mapred|hdfs}-site.xml) that are available in the
various '''conf/''' directories of your Hadoop installation needs to be on the '''CLASSPATH'''
of your Java application for it to get found and applied. Another way of ensuring that no
set configuration gets overridden by any Job is to set those properties as final; for example:

Mime
View raw message