reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yingda Chen <ydc...@gmail.com>
Subject Re: Improving yarn container allocation time
Date Tue, 21 Oct 2014 14:43:28 GMT
That is something we observed but did not have the time to dig into. Right
now for all evaluator requests initiated from .NET, we are forcing the rack
to be "*". Indeed it would be great to get the rack-specification working.

-Yingda

On Tue, Oct 21, 2014 at 5:39 AM, John (Youngseok) Yang <johnyangk@gmail.com>
wrote:

> HI,
>
>
> Sorry for the delay. I am struggling with rack configurations of the YARN
> cluster I am testing my code in.
>
> The YARN RM does not respond to rack-specific container request with
> relax_locality=false. (even for the unmodified REEF master branch code)
>
> Hopefully I can get it to work properly(or find another way to do this) and
> test my code in this week.
>
>
> Best regards,
> John
>
>
> On Fri, Oct 17, 2014 at 2:43 PM, John (Youngseok) Yang <
> johnyangk@gmail.com>
> wrote:
>
> > Hello all,
> >
> >
> > My name is John. I am a graduate student at SNU.
> >
> > With help from Professor Chun, Markus, Brian, and Yingda, I am working on
> > faster yarn container allocation.
> >
> > To circumvent YARN-314 and subtle concurrency issues wrt AMRMClientAsync,
> > currently REEF makes one-by-one container requests to YARN RM. It takes 2
> > seconds per request.
> >
> > I got my branch to the point where it does faster "one kind(same resource
> > vector) at a time" requests for which the "relax_locality" condition is
> > true.
> >
> > This is the url of my repo:
> > https://github.com/johnyangk/REEF/compare/jyang-improve-yarn-allocation
> >
> > For the next few days I am going to work on the locality-specific
> > requests. If I have any questions along the way, I'll use this thread.
> >
> >
> > Thanks,
> > John
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message