hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcin Sieniek <msien...@wirtualne-tatry.pl>
Subject Re: Is MapReduce NFS friendly
Date Mon, 10 May 2010 18:39:55 GMT
Hi Jyothish,

I had exactly the same problem and I solved it. To answer your question: 
as for me, HDFS and NFS are totally incompatible;) However, you may 
configure MadReduce to run on NFS only, without HDFS. See the last but 
one post here:
http://old.nabble.com/Hadoop-over-Lustre--td19092864.html
I did it and it works very well for NFS too (note that old 
hadoop-site.xml was splited to core-site.xml, mapred-site.xml and 
hdfs-site.xml in newer releases). Let me know if you have any problems 
with this configuration.

Marcin

Le 2010-05-10 20:16, alex kamil a écrit :
> Jyothish,
>
> as far as i know it is not recommended to run Hadoop on NFS, you 
> suppose to use use local volumes for all mapred and dfs directories
>
> Alex
>
> On Mon, May 10, 2010 at 2:00 PM, Jyothish Soman 
> <jyothish.soman@gmail.com <mailto:jyothish.soman@gmail.com>> wrote:
>
>     I have a distributed system on NFS, and wanted to use MapReduce on
>     it, but the system keeps spawning errors related to inability to
>     allocate temporary space.
>     Though sufficient is available, hence my question.
>      Is HDFS and NFS compatible?.
>
>


Mime
View raw message