reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John (Youngseok) Yang" <>
Subject Re: Improving yarn container allocation time
Date Tue, 21 Oct 2014 12:39:24 GMT

Sorry for the delay. I am struggling with rack configurations of the YARN
cluster I am testing my code in.

The YARN RM does not respond to rack-specific container request with
relax_locality=false. (even for the unmodified REEF master branch code)

Hopefully I can get it to work properly(or find another way to do this) and
test my code in this week.

Best regards,

On Fri, Oct 17, 2014 at 2:43 PM, John (Youngseok) Yang <>

> Hello all,
> My name is John. I am a graduate student at SNU.
> With help from Professor Chun, Markus, Brian, and Yingda, I am working on
> faster yarn container allocation.
> To circumvent YARN-314 and subtle concurrency issues wrt AMRMClientAsync,
> currently REEF makes one-by-one container requests to YARN RM. It takes 2
> seconds per request.
> I got my branch to the point where it does faster "one kind(same resource
> vector) at a time" requests for which the "relax_locality" condition is
> true.
> This is the url of my repo:
> For the next few days I am going to work on the locality-specific
> requests. If I have any questions along the way, I'll use this thread.
> Thanks,
> John

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message