Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of matei@eecs.berkeley.edu
 designates 169.229.60.87 as permitted sender)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1077)
Subject: Re: Small doubt in MR
From: Matei Zaharia <matei@eecs.berkeley.edu>
In-Reply-To: <c9b0d8bd1001022008u77b61716u7c748c6d18c09a06@mail.gmail.com>
Date: Sat, 2 Jan 2010 23:15:54 -0500
Content-Transfer-Encoding: quoted-printable
Message-Id: <E9767BDD-4003-47B1-8FBD-89CAF69C068E@eecs.berkeley.edu>
References: <73d592f61001022004l41495303ne841c44ed51ab76d@mail.gmail.com>
 <c9b0d8bd1001022008u77b61716u7c748c6d18c09a06@mail.gmail.com>
To: common-user@hadoop.apache.org

If you want the code to happen on only one machine, why not run it in =
your driver program that submits the MapReduce job?

You could also create a special input record that tells the mapper who =
gets that record that it's the chosen one. However, note that that =
mapper may be run multiple times due hardware failures.

Matei

On Jan 2, 2010, at 11:08 PM, Mark Kerzner wrote:

> I think you need some kind of semaphore that you can turn on by the =
first
> reducer. For example, allocating a file in HDFS would work - if you =
could
> guarantee that it is an atomic operation (create-if-does-not-exist).
>=20
> Mark
>=20
> On Sat, Jan 2, 2010 at 10:04 PM, bharath v <
> bharathvissapragada1990@gmail.com> wrote:
>=20
>> Hi,
>>=20
>> I want a particular "section of code" to run only in any "ONE" of the
>> mappers . So I employed the following procedure.
>>=20
>> Main-Class
>> {
>>=20
>>       public boolean flag =3D true;
>>=20
>>       Map-Class
>>      {
>>            if(flag)
>>           {
>>=20
>>               flag=3Dfalse;
>>              /* section of code */
>>           }
>>=20
>>=20
>> }
>>=20
>> I am running this code on in pseudo-distributed mode and its working =
fine .
>> I doubt whether this runs correctly in distributed mode because , =
mappers
>> on
>> other systems have to notified of the changed "flag" .. Any Comments =
? If
>> this is wrong , any suggestions on what method I must follow to =
achieve
>> this
>> functionality in D-mode .
>>=20
>> Thanks
>>=20