reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dylan Hutchison <dhutc...@cs.washington.edu>
Subject Re: Tang in Yarn - overriding a NamedParameter
Date Fri, 05 Aug 2016 22:07:30 GMT
Hi John,

The exception occurs after the evaluator spawns.  I think you have a point:
Reef may be working properly, but there may be other problems with the
application.

For now I got around the issue by installing HDFS and switching the file
system to that.  The directory "/tmp" is globally visible on HDFS, so
everything worked there.  It's a shame I couldn't make it work on the
Lustre file system (in which /tmp is *not* mounted), but I think this is an
application problem instead of a Reef problem.

Cheers, Dylan

On Wed, Aug 3, 2016 at 11:28 PM, John Yang <johnyangk@gmail.com> wrote:

> Hi Dylan,
>
> Does the exception occur before the Evaluator process is spawned or after?
> If it's before, then I think YARN is failing to locate the directory.
> If it's after, I really don't have a good idea as the parameter seem to be
> for Driver only, being in the package "org.apache.reef.driver.parameters".
>
> Thanks,
> John
>
>
> On Wed, Aug 3, 2016 at 4:55 AM, Dylan Hutchison <
> dhutchis@cs.washington.edu>
> wrote:
>
> > Thanks Gyewon and John.  I made some progress: the driver now sets the
> > DriverJobSubmissionDirectory
> > <
> > https://reef.apache.org/apidocs/0.15.0/org/apache/
> reef/javabridge/generic/Launch.DriverJobSubmissionDirectory.html
> > >
> > to the appropriate value and the Application Master recognizes it.
> >
> > Unfortunately, this information does not propagate to the containers that
> > the Driver requests, when it requests an Evaluator.  The containers
> > continue to use the default value "/tmp".  The message the driver
> receives
> > from the container for an allocated evaluator is a FileNotFoundException,
> > because the evaluator tries to use the wrong directory and can't find the
> > jars that Reef is trying to send to the evaluators.
> >
> > Is there a way to propogate the DriverJobSubmissionDirectory globally to
> > all Yarn processes?  I've looked at the EvaluatorRequest
> > <
> > https://reef.apache.org/apidocs/0.15.0/org/apache/reef/driver/evaluator/
> EvaluatorRequest.html
> > >
> > API but can't find anything.
> >
> > Thanks, Dylan
> >
> >
> > On Mon, Aug 1, 2016 at 11:40 PM, Gyewon Lee <strayyyyyy@gmail.com>
> wrote:
> >
> > > Hi, Dylan.
> > >
> > > In Tang NamedParameter, "default_value" is ignored when you explicitly
> > set
> > > your own value via JavaConfigurationBuilder (using methods like
> > > bindNamedParameter()) or ConfigurationModule (using method 'set()' in
> > > John's answer). So, you don't need to modify REEF source code for
> > changing
> > > default_value. In your case, John's solution will work quite nicely
> > because
> > > there is a predefined ConfigurationModule
> (YarnDriverConfiguration.CONF).
> > >
> > > In other cases where the ConfigurationModule is not set for your case,
> > then
> > > you can bind the value for NamedParameters, like
> > >
> > > JavaConfigurationBuilder cb =
> > > Tang.Factory.getTang().newConfigurationBuilder();
> > > cb.bindNamedParameter(JobSubmissionDirectoryPrefix.class,
> > > "/path/to/job/submission/directory/prefix");
> > > existingConfiguration.merge(cb.build());
> > >
> > > You don't need to do like that in this case, but I hope this would help
> > in
> > > your future REEF programming.
> > >
> > > Best regards,
> > > Gyewon Lee
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message