Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:date:from:user-agent:mime-version:to:cc:subject:
	references:in-reply-to:content-type:content-transfer-encoding;
	b=Oon4VM38dT3GgdqJo6S+OH1a5XdHtPy0g2aX0H1/Eh6Lhq6Da2/Afq+m5DDpenyR
Message-ID: <4C11B3CB.1080303@yahoo-inc.com>
Date: Fri, 11 Jun 2010 09:25:55 +0530
From: Vinod KV <vinodkv@yahoo-inc.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.9.1.9) Gecko/20100423 Lightning/1.0b1 Thunderbird/3.0.4
MIME-Version: 1.0
To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
CC: Saurabh Agarwal <srbh.grwl@gmail.com>
Subject: Re: job conf object
References: <AANLkTimL5QigrXr8c8zL_1K8-VWVZ6N-JASzu0rBKsg_@mail.gmail.com>
In-Reply-To: <AANLkTimL5QigrXr8c8zL_1K8-VWVZ6N-JASzu0rBKsg_@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On Wednesday 26 May 2010 04:38 PM, Saurabh Agarwal wrote:
> Hii..
> I am toying around with hadoop configuration.
> I am trying to replace HDFS with a common nfsmount, I only have map tasks.
> so intermediate outputs need not be communicated!!!
> So I want is there a way to make the temp directory local to the nodes and
> place job conf object and jar in a nfs mount so all the nodes can access
> it..
> Saurabh Agarwal
>    

Ideally you can do it, because MapReduce uses FileSystem APIs 
everywhere, but you may find some quirks.

OTOH, it is a very very bad idea and highly discouraged to run MapReduce 
on NFS - as soon as the number of nodes and thus tasks scales up, NFS 
will become bottlenecked and tasks/jobs will start failing with hard to 
debug failures.

+Vinod