Mailing-List: contact hadoop-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <20070731153623.6419jivmucckk8wk@tern.doc.ic.ac.uk>
Date: Tue, 31 Jul 2007 15:36:23 +0100
From: ojh06@doc.ic.ac.uk
To: hadoop-user@lucene.apache.org
Subject: Re: Error reporting from map function
References: <20070730214209.keqxk01140kococg@tern.doc.ic.ac.uk>
	<20070730214414.GB5688@killa.net>
	<68d348060707310416p73ab0818pc4efdaa2b9c0dc8f@mail.gmail.com>
In-Reply-To: <68d348060707310416p73ab0818pc4efdaa2b9c0dc8f@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset=ISO-8859-1;
	DelSp="Yes";
	format="flowed"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
User-Agent: Internet Messaging Program (IMP) H3 (4.1.3)

Well, I don't think it will be too much of a problem for me. I'll only  
be running this one type of job. The problem I have is that I can only  
throw IOExceptions out of the Mapper function. So if a job fails for  
some other reason, other than my numerical calculation error I have no  
way of knowing. I'd like to retry if its a communication problem, but  
terminate if its a calculation problem within my function.

I'm getting the feeling this isn't possible?

Quoting Jeroen Verhagen <jeroenverhagen@gmail.com>:

> Hi,
>
> On 7/30/07, Anthony D. Urso <anthonyu@killa.net> wrote:
>> Call JobConf.setMaxMapAttempts(0) in the job conf, then throw an exception
>> when your mapper fails.  This should kill the entire job instantly, since
>> the job tracker will allow no mapper failures.
>
> Wouldn't this cause all other running and future jobs to stop
> attempting to recover from an error? Or do all jobs have copies of the
> original job conf?
>
> --
>
> regards,
>
> Jeroen
>