Mailing-List: contact dev-help@systemml.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.apache.org
MIME-Version: 1.0
From: Matthias Boehm <mboehm7@googlemail.com>
Date: Wed, 25 Oct 2017 13:48:53 -0700
Message-ID: <CAOyjQwHGW5_9Fhow=Vem-Z9AzxPwhS8rMJy6o-NG4_6AyjmiQw@mail.gmail.com>
Subject: Re: Get plans before and after rewrites
To: dev@systemml.apache.org
Content-Type: multipart/alternative; boundary="089e08e4b65dffcd9e055c652e32"
archived-at: Wed, 25 Oct 2017 20:48:59 -0000

--089e08e4b65dffcd9e055c652e32
Content-Type: text/plain; charset="UTF-8"

Hi Nantia,

sure, let me clarify these two statements:

1) Optimization level: When I said O1, I was referring to SystemML's
configurable optimization level 1, which can be set through
'sysml.optlevel' (or 'optlevel' in SystemML 0.14) in a SystemML-config.xml
file or via setConfigProperty in MLContext. By default we use O2 with all
rewrites enabled, in O1, certain rewrites are disabled, which allows
comparing plans with and without rewrites. Furthermore, the quoted
statement below was more of a note for myself, to cleanup this handling of
remaining rewrites (that are not conditionally disabled yet) in an upcoming
SystemML release.

2) Log4j Configuration: You can take the log4j properties file from our
conf directory, put it into your classpath, and append the following line
to this file (alternatively with TRACE instead of DEBUG if you want to see
more details):
log4j.logger.org.apache.sysml.hops.rewrite=DEBUG

Regards,
Matthias

On Wed, Oct 25, 2017 at 4:24 AM, Nantia Makrynioti <nantiamakr@gmail.com>
wrote:

> Hello Matthias,
>
> Thanks a lot for replying and sorry for my late response.
>
> I have two more questions regarding the steps you described.
>
>  I think it's a
> > good idea to clean this up (disable these remaining rewrites in O1 as
> > well), which would be useful for debugging.
>
>
> I don't understand what you mean by O1. So how can I disable these
> remaining rewrites?
>
> If you're debugging any specific issue and want to see which rewrites
> > trigger where, you can set the log level for package
> > 'org.apache.sysml.hops.rewrite' to DEBUG or TRACE in your log4j
> > configuration.
>
>
> I am working with SystemML 0.14.0. There is a log4j.properties file, but I
> cannot find anything relevant to  'org.apache.sysml.hops.rewrite'. Is
> there
> another file I should check?
>
> Thanks again,
> Nantia
>
> 2017-10-13 23:29 GMT+03:00 Matthias Boehm <mboehm7@googlemail.com>:
>
> > Hi Nantia,
> >
> > in optimization level 1, we disable the following rewrites and the
> explain
> > hops or runtime output will show the resulting plan:
> > * Disable common-subexpression elimination
> > * Disable algebraic simplifications (static and dynamic)
> > * Disable inter-procedural analysis
> > * Disable branch removal and statement block merge
> > * Disable sum-product rewrites
> > * Disable update-in-place rewrites
> >
> > So, to compare plans without and with rewrites, you would need to run
> with
> > optimization level 1 and 2 (our default), and capture & compare these
> > outputs.
> >
> > Note that there are a number of rewrites such as matrix multiplication
> > chain optimization, operator selection, hop-lop rewrites that are always
> > applied and thus not affected by the optimization level. I think it's a
> > good idea to clean this up (disable these remaining rewrites in O1 as
> > well), which would be useful for debugging.
> >
> > If you're debugging any specific issue and want to see which rewrites
> > trigger where, you can set the log level for package
> > 'org.apache.sysml.hops.rewrite' to DEBUG or TRACE in your log4j
> > configuration. If you're compiling from sources, you can also simply set
> > ProgramRewriter.LDEBUG to true to do the same thing.
> >
> > Regards,
> > Matthias
> >
> > On Fri, Oct 13, 2017 at 2:04 AM, Nantia Makrynioti <nantiamakr@gmail.com
> >
> > wrote:
> >
> > > Hello,
> > >
> > > I set optimization level to 1 in SystemML-config.xml, in order to get
> HOP
> > > and LOP plans before and after rewrites. However, I am still getting
> > just a
> > > single plan.
> > >
> > > I am using spark shell to execute the dml script.
> > >
> > > Thank you in advance,
> > > Nantia
> > >
> >
>

--089e08e4b65dffcd9e055c652e32--