hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Sammer <e...@lifeless.net>
Subject Re: mapred.system.dir
Date Fri, 12 Feb 2010 14:23:35 GMT
On 2/12/10 8:40 AM, Edson Ramiro wrote:
> Hi all,
> I'm setting up a Hadoop Cluster and some doubts have
>  arisen about hadoop configuration.
> The Hadoop Cluster Setup [1] says that the mapred.system.dir must
> be in the HDFS and be accessible from both the server and clients.
> Where is the HDFS directory? is the dfs.data.dir?
> should I export by NFS or other protocol the mapred.system.dir to
> leave it accessible from server and clients?
> Thanks in advance
> [1] http://hadoop.apache.org/common/docs/current/cluster_setup.html
> Edson Ramiro


An HDFS file system is a distributed global view controlled by the
namenode. If a file is "in HDFS" all clients and servers that are
pointed at the namenode will be able to see everything. This means that
you don't need to do anything special to export or reveal the
mapred.system.dir; that's what HDFS does. It's worth reading the HDFS
Architecture paper on the Hadoop site or the Google GFS paper for
details on how this all works and how it relates to map reduce.

Eric Sammer

View raw message