hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: Node scheduling in 2.1.x
Date Fri, 06 Sep 2013 06:43:46 GMT
That's right.  Nodes keep checking in and, when they do, the
ResourceManager looks for outstanding requests.  This means that assignment
of containers to nodes depends on the order that they heartbeat in.  If
container requests come in for specific nodes locality is achieved through
delay scheduling - the ResourceManager will wait for a configurable number
of heartbeats before assigning a container to a non-local node.  If strict
locality is turned on, the ResourceManager will wait indefinitely for a
local node.

-Sandy


On Fri, Sep 6, 2013 at 3:33 PM, hilfi alkaff <hilfialkaff@gmail.com> wrote:

> I see. What I'm wondering about is; when an application master tries to
> request a container from resource manager, which part of the code in the
> resource manager actually decide which node to fetch this container from.
> Is this step being done asynchronously (ie: Nodes keep checking if there
> are requests from the ResourceManager during the node update event?)
>
>
> On Fri, Sep 6, 2013 at 1:22 AM, Sandy Ryza <sandy.ryza@cloudera.com>
> wrote:
>
> > Hi Hilfi,
> >
> > Nodes are constantly heartbeating to the ResourceManager.  A node update
> > event is triggered each time this happens.
> >
> > -Sandy
> >
> >
> > On Fri, Sep 6, 2013 at 3:20 PM, hilfi alkaff <hilfialkaff@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'm trying to trace the code flow on the scheduling done in YARN. I
> would
> > > like to know where the code that does which node to schedule for the
> > jobs.
> > >
> > > I found the handle() function in the resource manager's scheduler (eg:
> > > CapacityScheduler.java) that handles node update event which then
> > executes
> > > the assignment of containers for that particular node, but I do not
> > > understand how that node even get chosen.
> > >
> > > If anybody could tell me about a file, function or module name that
> does
> > > this, that would be extremely helpful.
> > >
> > > --
> > > ~Hilfi Alkaff~
> > >
> >
>
>
>
> --
> ~Hilfi Alkaff~
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message