reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobin Baker <tdba...@cs.washington.edu>
Subject Re: using NamedParameter to implement a no-arg boolean command-line option
Date Fri, 04 Dec 2015 17:53:39 GMT
The need is simple: if my launcher is invoked by "yarn jar", I should use
the YARN runtime, and the local runtime otherwise. I was thinking of
testing for YARN configuration using the YarnConfiguration default
constructor, but that seems like a fragile hack. Currently my driver- or
evaluator-based code doesn't need to know which runtime is already in use,
but I could see that being important for cases where REEF doesn't expose
some YARN functionality (such as, until recently, node/rack locality). I
imagine Mesos users might have similar needs. I like your idea of a
"runtime string" exposed by appropriate interfaces in the
client/driver/evaluators.


On Fri, Dec 4, 2015 at 9:26 AM, Markus Weimer <markus@weimo.de> wrote:

> On 2015-12-03 18:54, Tobin Baker wrote:
>
>> I'm looking for a way to detect my code is running under YARN
>>
>
> Can you elaborate on where that need is coming from? In an ideal world,
> your app wouldn't have to know. Of course, no one lives in that world :-)
>
> I can imagine a few hacky ways to do this, e.g. by using multiple
> constructors, some of which can only be invoked by Tang if YARN is
> configured. I would advise not to do that :-)
>
> If we wanted to make this a first class thing in REEF, we could have the
> concept of a "Runtime String" in much the same way of a user agent in
> browsers. That would allow us to keep the set of runtimes open while
> also giving users access to it.
>
> On the client, we could expose a new method in
> `org.apache.reef.client.REEF` to get access to it. On the Driver, we'd
> probably want to create a new interface `RuntimeInformation` or such.
>
> Markus
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message