Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <488F315C.4010105@nbi.dk>
Date: Tue, 29 Jul 2008 17:03:56 +0200
From: =?UTF-8?B?Q2hyaXN0aWFuIFVscmlrIFPDuHR0cnVw?= <soettrup@nbi.dk>
User-Agent: Thunderbird 2.0.0.16 (Windows/20080708)
MIME-Version: 1.0
To: core-user@hadoop.apache.org
Subject: Re: iterative map-reduce
References: <643aa4870806260443i17bf8a7bp6451b849b95ba1e@mail.gmail.com>
	 <6FC55D62-A507-4D55-B953-AA371FB06954@cs.utexas.edu>
	 <826304a60807290710j628237c0oaf3c009f252731a3@mail.gmail.com>
	 <D4D20015-0F43-4446-9DE1-48D689D3B58A@cis.upenn.edu>
 <826304a60807290738y54be709bjbd4581560f3d1b81@mail.gmail.com>
In-Reply-To: <826304a60807290738y54be709bjbd4581560f3d1b81@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Shirley,

I am basically doing as Qin suggested.
I am running a job iteratively until some condition is met.
My main looks something like:(in pseudo code)

main:
 while (!converged):
   make new jobconf
   setup jobconf
   run jobconf
   check reporter for statistics
   decide if converged

I use a custom reporter to check on the fitness of the solution in the 
reduce phase.

If you need more(real java) code drop me a line.

Cheers,
Christian

Qin Gao wrote:
> I think it is nothing to do with the framework, just treat the mapredcue as
> a batch process or a subroutine, and you may iteratively call them. If there
> are such interface, I am also interested to know.
>
>
>
> On Tue, Jul 29, 2008 at 10:31 AM, Shirley Cohen <shirleyc@cis.upenn.edu>wrote:
>
>   
>> Thanks... would the iterative script be run outside of Hadoop? I was
>> actually trying to figure out if the framework could handle iterations.
>>
>> Shirley
>>
>>
>> On Jul 29, 2008, at 9:10 AM, Qin Gao wrote:
>>
>>  if you are using java, just create job configure again and run it,
>>     
>>> otherwise
>>> you just need to write a iterative script.
>>>
>>> On Tue, Jul 29, 2008 at 9:57 AM, Shirley Cohen <scohen@cs.utexas.edu>
>>> wrote:
>>>
>>>  Hi,
>>>       
>>>> I want to call a map-reduce program recursively until some condition is
>>>> met.  How do I do that?
>>>>
>>>> Thanks,
>>>>
>>>> Shirley
>>>>
>>>>
>>>>
>>>>         
>
>