systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nantia Makrynioti <nantiam...@gmail.com>
Subject Re: Get plans before and after rewrites
Date Thu, 26 Oct 2017 14:11:27 GMT
Thank you for clarifying this, Matthias!

2017-10-25 23:48 GMT+03:00 Matthias Boehm <mboehm7@googlemail.com>:

> Hi Nantia,
>
> sure, let me clarify these two statements:
>
> 1) Optimization level: When I said O1, I was referring to SystemML's
> configurable optimization level 1, which can be set through
> 'sysml.optlevel' (or 'optlevel' in SystemML 0.14) in a SystemML-config.xml
> file or via setConfigProperty in MLContext. By default we use O2 with all
> rewrites enabled, in O1, certain rewrites are disabled, which allows
> comparing plans with and without rewrites. Furthermore, the quoted
> statement below was more of a note for myself, to cleanup this handling of
> remaining rewrites (that are not conditionally disabled yet) in an upcoming
> SystemML release.
>
> 2) Log4j Configuration: You can take the log4j properties file from our
> conf directory, put it into your classpath, and append the following line
> to this file (alternatively with TRACE instead of DEBUG if you want to see
> more details):
> log4j.logger.org.apache.sysml.hops.rewrite=DEBUG
>
> Regards,
> Matthias
>
> On Wed, Oct 25, 2017 at 4:24 AM, Nantia Makrynioti <nantiamakr@gmail.com>
> wrote:
>
> > Hello Matthias,
> >
> > Thanks a lot for replying and sorry for my late response.
> >
> > I have two more questions regarding the steps you described.
> >
> >  I think it's a
> > > good idea to clean this up (disable these remaining rewrites in O1 as
> > > well), which would be useful for debugging.
> >
> >
> > I don't understand what you mean by O1. So how can I disable these
> > remaining rewrites?
> >
> > If you're debugging any specific issue and want to see which rewrites
> > > trigger where, you can set the log level for package
> > > 'org.apache.sysml.hops.rewrite' to DEBUG or TRACE in your log4j
> > > configuration.
> >
> >
> > I am working with SystemML 0.14.0. There is a log4j.properties file, but
> I
> > cannot find anything relevant to  'org.apache.sysml.hops.rewrite'. Is
> > there
> > another file I should check?
> >
> > Thanks again,
> > Nantia
> >
> > 2017-10-13 23:29 GMT+03:00 Matthias Boehm <mboehm7@googlemail.com>:
> >
> > > Hi Nantia,
> > >
> > > in optimization level 1, we disable the following rewrites and the
> > explain
> > > hops or runtime output will show the resulting plan:
> > > * Disable common-subexpression elimination
> > > * Disable algebraic simplifications (static and dynamic)
> > > * Disable inter-procedural analysis
> > > * Disable branch removal and statement block merge
> > > * Disable sum-product rewrites
> > > * Disable update-in-place rewrites
> > >
> > > So, to compare plans without and with rewrites, you would need to run
> > with
> > > optimization level 1 and 2 (our default), and capture & compare these
> > > outputs.
> > >
> > > Note that there are a number of rewrites such as matrix multiplication
> > > chain optimization, operator selection, hop-lop rewrites that are
> always
> > > applied and thus not affected by the optimization level. I think it's a
> > > good idea to clean this up (disable these remaining rewrites in O1 as
> > > well), which would be useful for debugging.
> > >
> > > If you're debugging any specific issue and want to see which rewrites
> > > trigger where, you can set the log level for package
> > > 'org.apache.sysml.hops.rewrite' to DEBUG or TRACE in your log4j
> > > configuration. If you're compiling from sources, you can also simply
> set
> > > ProgramRewriter.LDEBUG to true to do the same thing.
> > >
> > > Regards,
> > > Matthias
> > >
> > > On Fri, Oct 13, 2017 at 2:04 AM, Nantia Makrynioti <
> nantiamakr@gmail.com
> > >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I set optimization level to 1 in SystemML-config.xml, in order to get
> > HOP
> > > > and LOP plans before and after rewrites. However, I am still getting
> > > just a
> > > > single plan.
> > > >
> > > > I am using spark shell to execute the dml script.
> > > >
> > > > Thank you in advance,
> > > > Nantia
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message