reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John (Youngseok) Yang" <johnya...@gmail.com>
Subject Re: Improving yarn container allocation time
Date Tue, 21 Oct 2014 12:39:24 GMT
HI,


Sorry for the delay. I am struggling with rack configurations of the YARN
cluster I am testing my code in.

The YARN RM does not respond to rack-specific container request with
relax_locality=false. (even for the unmodified REEF master branch code)

Hopefully I can get it to work properly(or find another way to do this) and
test my code in this week.


Best regards,
John


On Fri, Oct 17, 2014 at 2:43 PM, John (Youngseok) Yang <johnyangk@gmail.com>
wrote:

> Hello all,
>
>
> My name is John. I am a graduate student at SNU.
>
> With help from Professor Chun, Markus, Brian, and Yingda, I am working on
> faster yarn container allocation.
>
> To circumvent YARN-314 and subtle concurrency issues wrt AMRMClientAsync,
> currently REEF makes one-by-one container requests to YARN RM. It takes 2
> seconds per request.
>
> I got my branch to the point where it does faster "one kind(same resource
> vector) at a time" requests for which the "relax_locality" condition is
> true.
>
> This is the url of my repo:
> https://github.com/johnyangk/REEF/compare/jyang-improve-yarn-allocation
>
> For the next few days I am going to work on the locality-specific
> requests. If I have any questions along the way, I'll use this thread.
>
>
> Thanks,
> John
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message