hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arinto Murdopo <ari...@gmail.com>
Subject Re: Question regarding "attempt" concept in YARN source code
Date Fri, 09 Nov 2012 14:27:23 GMT
Thank you very much Tom!



Arinto Murdopo
European Master in Distributed Computing (EMDC)
Universitat Politècnica de Catalunya · BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Phone: +46 725 548 759

On Fri, Nov 9, 2012 at 1:25 AM, Thomas Graves <tgraves@yahoo-inc.com> wrote:

> With YARN you now have applications. This is how the RM (resource manager)
> views things.  MapReduce is one type of application.  An RMAppAttempt is
> an attempt to run one particular application. During the execution of that
> application something might go wrong which causes it to die before the
> application finishes.  For instance the node the application master is on
> dies.   The RM will then try to run that application again - a second
> attempt. There is a configuration that controls how many attempts to run
> that application you make before giving up.  You application can chose to
> handle re-attempts. For instance the MapReduce application knows its a
> second (or third or etc..) attempt, reads the job history file and only
> re-runs the tasks that haven't finished.
> A task-attempt is at the MapReduce application level.  A Task in this case
> is either a map or a reduce task.  An attempt again is just one attempt at
> running a single map or reduce.  That single attempt can fail for lots of
> reasons - host goes down, etc.  The mapreduce application master will
> rerun that task (thus a second attempt) up to whatever is configured.
> Tom
> On 11/8/12 4:22 PM, "Arinto Murdopo" <arinto@gmail.com> wrote:
> >Hi guys,
> >
> >I have several questions related to YARN source code:
> >1. what is "task-attempt" in the this document (
> >https://issues.apache.org/jira/secure/attachment/12486023/MapR) ?
> >2. what is RMAppAttempt ? the documentation in the code mentions that
> >RMApp
> >may have multiple RMAppAttempt but I still could not reason what is
> >RMAppAttempt and why do we need this.
> >
> >regards,
> >
> >Arinto Murdopo
> >European Master in Distributed Computing (EMDC)
> >Universitat Politècnica de Catalunya · BarcelonaTech, Barcelona, Spain
> >KTH Royal Institute of Technology, Stockholm, Sweden
> >Phone: +46 725 548 759

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message