Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.214.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAE636z8fAcBpM9GVbaCK5TOLvbZG3n5mfR40ARNupo-CamrLoA@mail.gmail.com>
References: 
 <CAE636z_Dd9AkDupCyUmSZ-RuaxUvw9RU8SKCn8QVNsXkxMhRiA@mail.gmail.com>
 <5B24054F-762B-43EA-824F-9E0641B84584@123.org>
 <CAE636z8fAcBpM9GVbaCK5TOLvbZG3n5mfR40ARNupo-CamrLoA@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Fri, 14 Sep 2012 12:54:41 +0530
Message-ID: 
 <CAOcnVr1Wpgt9Gpj2=DUC+8oiyZCotiNPcbkNvjxRmEegJXfLRA@mail.gmail.com>
Subject: Re: What's the basic idea of pseudo-distributed Hadoop ?
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi Jason,

I think you're confusing the standalone mode with a pseudo-distributed
mode. The former is a limited mode of MR where no daemons need to be
deployed and the tasks run in a single JVM (via threads).

A pseudo distributed cluster is a cluster where all daemons are
running on one node itself. Hence, not "distributed" in the sense of
multi-nodes (no use of an network gear) but works in the same way
between nodes (RPC, etc.) as a fully-distributed one.

If an MR program works fine in a pseudo-distributed mode, it "should"
work (no guarantee) fine in a fully-distributed mode iff all nodes
have the same arch/OS, same JVM, and job-specific configurations. This
is because tasks execute on various nodes and may be affected by the
node's behavior or setup that is different from others - and thats
something you'd have to detect/know about if it exhibits failures more
than others.

On Fri, Sep 14, 2012 at 11:58 AM, Jason Yang <lin.yang.jason@gmail.com> wrote:
> Hey, Kai
>
> Thanks for you reply.
>
> I was wondering what's difference btw the pseudo-distributed and
> fully-distributed hadoop, except the maximum number of map/reduce.
>
> And if a MR program works fine in pseudo-distributed cluster, will it work
> exactly fine in the fully-distributed cluster ?
>
>
> 2012/9/14 Kai Voigt <k@123.org>
>>
>> e default setting is that a tasktracker can run up to two map and reduce
>> tasks in parallel (mapred.tasktracker.map.tasks.maximum and
>> mapred.tasktracker.reduce.tasks.maximum), so you will actually see some
>> concurrency on your one machine.
>
>
>
>
> --
> YANG, Lin
>


-- 
Harsh J