reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John (Youngseok) Yang" <johnya...@gmail.com>
Subject Re: Improving yarn container allocation time
Date Wed, 22 Oct 2014 13:18:41 GMT
I filed a PR at https://github.com/Microsoft-CISL/REEF/pull/1024.

Thanks,
John

On Wed, Oct 22, 2014 at 9:42 AM, John (Youngseok) Yang <johnyangk@gmail.com>
wrote:

> Hi,
>
>
> I see. Then I'll treat rack-locality as a separate issue and tackle it
> later.
>
> My username is johnyangk.
>
>
> Thanks,
> John
>
> On Wed, Oct 22, 2014 at 2:03 AM, Markus Weimer <weimer@apache.org> wrote:
>
>> Hi,
>>
>> I have created a JIRA for this:
>>
>> https://issues.apache.org/jira/browse/REEF-6
>>
>> John, what is your JIRA user name? That way I can assign it to you :-)
>>
>> Markus
>>
>>
>> On Tue, Oct 21, 2014 at 7:43 AM, Yingda Chen <ydchen@gmail.com> wrote:
>>
>> > That is something we observed but did not have the time to dig into.
>> Right
>> > now for all evaluator requests initiated from .NET, we are forcing the
>> rack
>> > to be "*". Indeed it would be great to get the rack-specification
>> working.
>> >
>> > -Yingda
>> >
>> > On Tue, Oct 21, 2014 at 5:39 AM, John (Youngseok) Yang <
>> > johnyangk@gmail.com>
>> > wrote:
>> >
>> > > HI,
>> > >
>> > >
>> > > Sorry for the delay. I am struggling with rack configurations of the
>> YARN
>> > > cluster I am testing my code in.
>> > >
>> > > The YARN RM does not respond to rack-specific container request with
>> > > relax_locality=false. (even for the unmodified REEF master branch
>> code)
>> > >
>> > > Hopefully I can get it to work properly(or find another way to do
>> this)
>> > and
>> > > test my code in this week.
>> > >
>> > >
>> > > Best regards,
>> > > John
>> > >
>> > >
>> > > On Fri, Oct 17, 2014 at 2:43 PM, John (Youngseok) Yang <
>> > > johnyangk@gmail.com>
>> > > wrote:
>> > >
>> > > > Hello all,
>> > > >
>> > > >
>> > > > My name is John. I am a graduate student at SNU.
>> > > >
>> > > > With help from Professor Chun, Markus, Brian, and Yingda, I am
>> working
>> > on
>> > > > faster yarn container allocation.
>> > > >
>> > > > To circumvent YARN-314 and subtle concurrency issues wrt
>> > AMRMClientAsync,
>> > > > currently REEF makes one-by-one container requests to YARN RM. It
>> > takes 2
>> > > > seconds per request.
>> > > >
>> > > > I got my branch to the point where it does faster "one kind(same
>> > resource
>> > > > vector) at a time" requests for which the "relax_locality"
>> condition is
>> > > > true.
>> > > >
>> > > > This is the url of my repo:
>> > > >
>> > https://github.com/johnyangk/REEF/compare/jyang-improve-yarn-allocation
>> > > >
>> > > > For the next few days I am going to work on the locality-specific
>> > > > requests. If I have any questions along the way, I'll use this
>> thread.
>> > > >
>> > > >
>> > > > Thanks,
>> > > > John
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message