hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Yarn resource request
Date Thu, 21 Nov 2013 10:12:41 GMT
On 21 November 2013 09:04, 牛兆捷 <nzjemail@gmail.com> wrote:

> Another question.
>
> For two request with the same priority, which one will be chosen?
>

No way to know. An equally interesting question is : how do you know which
one has been satisfied, and which one is oustanding?

I spent some time trying to do some of this for a service where I'd request
some instances on specific nodes, some anywhere, and tried to see if I
could determine which response satisified which request

https://github.com/hortonworks/hoya/blob/master/src/site/markdown/rolehistory.md

This all lives in a state library, which handles AMClientAsync events,
generates lists of actions, but doesn't talk to the AM itself

https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state

This is matched by an in-VM mock YARN service

https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/test/groovy/org/apache/hadoop/hoya/yarn/model/mock

I'd recommend this model-controller architecture purely for its testability

Returning to request tracking I ended up doing was keeping a map of
outstanding requests, and when they arrived I'd remove them from the list.
When every outstanding request had been satisfied, I'd clear the list of
any outstanding entries

https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state/OutstandingRequestTracker.java




>
>
> 2013/11/21 Bikas Saha <bikas@hortonworks.com>
>
> > That’s a known limitation of the current implementation of schedulers in
> > YARN. The API allows different requests of different sizes within the
> same
> > priority but the implementations of the schedulers do not support this.
> > For different sizes, one can either use different priorities or use the
> > highest size amongst the sizes within a priority.
> >
> > Bikas
> >
> > -----Original Message-----
> > From: 牛兆捷 [mailto:nzjemail@gmail.com]
> > Sent: Wednesday, November 20, 2013 7:44 PM
> > To: yarn-dev@hadoop.apache.org
> > Subject: Yarn resource request
> >
> > Currently, resource requests for the same priority and locality are
> > expected to all be the same size, how will Application Master do for
> > different size? use different requests?
> >
> > --
> > *Regards,*
> > *Zhaojie*
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
>
>
> --
> *Regards,*
> *Zhaojie*
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message