reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Yang <johnya...@gmail.com>
Subject Re: Tang in Yarn - overriding a NamedParameter
Date Fri, 05 Aug 2016 23:45:32 GMT
I see. Glad you found a solution. :)

John


On Sat, Aug 6, 2016 at 7:07 AM, Dylan Hutchison <dhutchis@cs.washington.edu>
wrote:

> Hi John,
>
> The exception occurs after the evaluator spawns.  I think you have a point:
> Reef may be working properly, but there may be other problems with the
> application.
>
> For now I got around the issue by installing HDFS and switching the file
> system to that.  The directory "/tmp" is globally visible on HDFS, so
> everything worked there.  It's a shame I couldn't make it work on the
> Lustre file system (in which /tmp is *not* mounted), but I think this is an
> application problem instead of a Reef problem.
>
> Cheers, Dylan
>
> On Wed, Aug 3, 2016 at 11:28 PM, John Yang <johnyangk@gmail.com> wrote:
>
> > Hi Dylan,
> >
> > Does the exception occur before the Evaluator process is spawned or
> after?
> > If it's before, then I think YARN is failing to locate the directory.
> > If it's after, I really don't have a good idea as the parameter seem to
> be
> > for Driver only, being in the package "org.apache.reef.driver.
> parameters".
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Aug 3, 2016 at 4:55 AM, Dylan Hutchison <
> > dhutchis@cs.washington.edu>
> > wrote:
> >
> > > Thanks Gyewon and John.  I made some progress: the driver now sets the
> > > DriverJobSubmissionDirectory
> > > <
> > > https://reef.apache.org/apidocs/0.15.0/org/apache/
> > reef/javabridge/generic/Launch.DriverJobSubmissionDirectory.html
> > > >
> > > to the appropriate value and the Application Master recognizes it.
> > >
> > > Unfortunately, this information does not propagate to the containers
> that
> > > the Driver requests, when it requests an Evaluator.  The containers
> > > continue to use the default value "/tmp".  The message the driver
> > receives
> > > from the container for an allocated evaluator is a
> FileNotFoundException,
> > > because the evaluator tries to use the wrong directory and can't find
> the
> > > jars that Reef is trying to send to the evaluators.
> > >
> > > Is there a way to propogate the DriverJobSubmissionDirectory globally
> to
> > > all Yarn processes?  I've looked at the EvaluatorRequest
> > > <
> > > https://reef.apache.org/apidocs/0.15.0/org/apache/
> reef/driver/evaluator/
> > EvaluatorRequest.html
> > > >
> > > API but can't find anything.
> > >
> > > Thanks, Dylan
> > >
> > >
> > > On Mon, Aug 1, 2016 at 11:40 PM, Gyewon Lee <strayyyyyy@gmail.com>
> > wrote:
> > >
> > > > Hi, Dylan.
> > > >
> > > > In Tang NamedParameter, "default_value" is ignored when you
> explicitly
> > > set
> > > > your own value via JavaConfigurationBuilder (using methods like
> > > > bindNamedParameter()) or ConfigurationModule (using method 'set()' in
> > > > John's answer). So, you don't need to modify REEF source code for
> > > changing
> > > > default_value. In your case, John's solution will work quite nicely
> > > because
> > > > there is a predefined ConfigurationModule
> > (YarnDriverConfiguration.CONF).
> > > >
> > > > In other cases where the ConfigurationModule is not set for your
> case,
> > > then
> > > > you can bind the value for NamedParameters, like
> > > >
> > > > JavaConfigurationBuilder cb =
> > > > Tang.Factory.getTang().newConfigurationBuilder();
> > > > cb.bindNamedParameter(JobSubmissionDirectoryPrefix.class,
> > > > "/path/to/job/submission/directory/prefix");
> > > > existingConfiguration.merge(cb.build());
> > > >
> > > > You don't need to do like that in this case, but I hope this would
> help
> > > in
> > > > your future REEF programming.
> > > >
> > > > Best regards,
> > > > Gyewon Lee
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message