reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashvin A <aas....@gmail.com>
Subject Re: Non-blocking version DriverLauncher.run
Date Mon, 20 Jun 2016 21:27:22 GMT
Hi,

I am restarting this thread to verify if my observation is as expected.

As Markus suggested, I borrowed code to configure client and exit when a
long running job starts on YARN. I am invoking reef.close() after receiving
RunningJob event and then exit the function. Invoking reef.close on client
does not kill the running job. However, I see this log:
"org.apache.reef.runtime.common.client.RunningJobsImpl
WARNING:  Force close job". This message is confusing as the job continues
to run on YARN. Should I ignore the log message or should I stop invoking
reef.close?

Thanks,
Ashvin

On Sat, Apr 9, 2016 at 8:17 AM, Markus Weimer <markus@weimo.de> wrote:

> Hi Ashvin,
>
> On 2016-04-08 17:12, Ashvin A wrote:
>
>> I am using DriverLauncher.run(config) to start a long running service.
>>
>
> Uh oh, that thing is getting a life of its own :)
>
>  Is there a non-blocking version of this method which returns
>> without killing the service.
>>
>
> Yes. DriverLauncher is actually a wrapper around our asynchronous API.
> That is the main API for the REEF Client, but it has proven to be quite
> hard to write integration tests and simple examples. Hence, we made
> `DriverLauncher` which offers a blocking call.
>
> If you want to make your own async client, the DriverLauncher source could
> be a good source of documentation. For example,
> `DriverLauncher.getLauncher()` contains the code to configure a client:
>
> ```
> final Configuration clientConfiguration = ClientConfiguration.CONF
>   .set(ClientConfiguration.ON_JOB_RUNNING, RunningJobHandler.class)
>   .set(ClientConfiguration.ON_JOB_COMPLETED, CompletedJobHandler.class)
>   .set(ClientConfiguration.ON_JOB_FAILED, FailedJobHandler.class)
>   .set(ClientConfiguration.ON_RUNTIME_ERROR, RuntimeErrorHandler.class)
>   .build();
> ```
>
> A REEF Client is very similar to a REEF Driver: It's a bunch of event
> handlers that REEF invokes. Once you fill out a `ClientConfiguration` like
> above, you merge it with the configuration of the runtime of choice and
> obtain a `REEF` instance. That instance can then be used to `.submit()` the
> DriverConfiguration to:
>
> ```
> Tang.Factory.getTang()
>   .newInjector(runtimeConfiguration, clientConfiguration)
>   .getInstance(REEF.class);
> ```
>
> `DriverLauncher` follows a slightly different pattern, where the
> `DriverLauncher` constructor gets the `REEF` instance. That has proven to
> be a useful pattern as it allows to pass other parameters besides the
> `REEF` instance.
>
> Please let me know how it goes. If the above works, we probably should
> turn it into a FAQ answer :)
>
> Markus
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message