reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John (Youngseok) Yang" <johnya...@gmail.com>
Subject Re: Improving yarn container allocation time
Date Wed, 22 Oct 2014 00:42:30 GMT
Hi,


I see. Then I'll treat rack-locality as a separate issue and tackle it
later.

My username is johnyangk.


Thanks,
John

On Wed, Oct 22, 2014 at 2:03 AM, Markus Weimer <weimer@apache.org> wrote:

> Hi,
>
> I have created a JIRA for this:
>
> https://issues.apache.org/jira/browse/REEF-6
>
> John, what is your JIRA user name? That way I can assign it to you :-)
>
> Markus
>
>
> On Tue, Oct 21, 2014 at 7:43 AM, Yingda Chen <ydchen@gmail.com> wrote:
>
> > That is something we observed but did not have the time to dig into.
> Right
> > now for all evaluator requests initiated from .NET, we are forcing the
> rack
> > to be "*". Indeed it would be great to get the rack-specification
> working.
> >
> > -Yingda
> >
> > On Tue, Oct 21, 2014 at 5:39 AM, John (Youngseok) Yang <
> > johnyangk@gmail.com>
> > wrote:
> >
> > > HI,
> > >
> > >
> > > Sorry for the delay. I am struggling with rack configurations of the
> YARN
> > > cluster I am testing my code in.
> > >
> > > The YARN RM does not respond to rack-specific container request with
> > > relax_locality=false. (even for the unmodified REEF master branch code)
> > >
> > > Hopefully I can get it to work properly(or find another way to do this)
> > and
> > > test my code in this week.
> > >
> > >
> > > Best regards,
> > > John
> > >
> > >
> > > On Fri, Oct 17, 2014 at 2:43 PM, John (Youngseok) Yang <
> > > johnyangk@gmail.com>
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > >
> > > > My name is John. I am a graduate student at SNU.
> > > >
> > > > With help from Professor Chun, Markus, Brian, and Yingda, I am
> working
> > on
> > > > faster yarn container allocation.
> > > >
> > > > To circumvent YARN-314 and subtle concurrency issues wrt
> > AMRMClientAsync,
> > > > currently REEF makes one-by-one container requests to YARN RM. It
> > takes 2
> > > > seconds per request.
> > > >
> > > > I got my branch to the point where it does faster "one kind(same
> > resource
> > > > vector) at a time" requests for which the "relax_locality" condition
> is
> > > > true.
> > > >
> > > > This is the url of my repo:
> > > >
> > https://github.com/johnyangk/REEF/compare/jyang-improve-yarn-allocation
> > > >
> > > > For the next few days I am going to work on the locality-specific
> > > > requests. If I have any questions along the way, I'll use this
> thread.
> > > >
> > > >
> > > > Thanks,
> > > > John
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message