Mailing-List: contact dev-help@systemml.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.apache.org
MIME-Version: 1.0
From: Matthias Boehm <mboehm7@googlemail.com>
Date: Fri, 16 Jun 2017 21:35:39 -0700
Message-ID: <CAOyjQwETPtnWDmNqH5FG-GZwKeg-EnL0zRVq03QLEtrc2vFmUg@mail.gmail.com>
Subject: Re: Parfor loop interdependencies
To: dev@systemml.apache.org
Content-Type: multipart/alternative; boundary="001a114db3aa12bb060552206f23"
archived-at: Sat, 17 Jun 2017 04:35:45 -0000

--001a114db3aa12bb060552206f23
Content-Type: text/plain; charset="UTF-8"

ok thanks for catching this Mike - I just gave it a try and the new IPA
propagated scalars into this function, which allowed for constant folding
of the indexing expressions, and effectively hid the underlying issue. With
disabled IPA, I'm able to reproduce this issue. Interestingly, it only
shows up in spark but not hadoop execution modes.

Regards,
Matthias

On Fri, Jun 16, 2017 at 5:35 PM, <dusenberrymw@gmail.com> wrote:

> On the latest master, versus the one from a few days ago, this issue
> appears to be "fixed".  The code is in PR 542, and the for-loop in the
> `nn/layers/conv2d_depthwise.dml::forward` function was the one that was
> previously resulting in null pointer exceptions when forced as a parfor
> loop.  The `check=0` is still needed, but the null pointer is no longer
> being thrown.
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Jun 14, 2017, at 9:08 PM, Matthias Boehm <mboehm7@googlemail.com>
> wrote:
> >
> > Generally, the parfor dependency analysis applies a series of tests
> > including traditional techniques from high-performance compilers combined
> > with additional rules for common cases. This dependency analysis tries to
> > proof that there are no loop-carried dependencies - so yes, false
> positives
> > can happen, in which case a user needs to take responsibility by
> disabling
> > the dependency analysis.
> >
> > Regarding your concrete problem - any null pointer exception is of course
> > bug and needs fixing. Could you please create a minimal scenario that
> > reproduces this issues and create a JIRA for it. I'm happy to help
> > resolving this.
> >
> > Regards,
> > Matthias
> >
> >> On Wed, Jun 14, 2017 at 4:24 PM, <dusenberrymw@gmail.com> wrote:
> >>
> >> While working on depthwise convolution, I wanted to make use of a parfor
> >> loop to run multiple convolutions at once (there is a vectorized
> approach
> >> as well, but that is out of scope for this question), but I'm running
> into
> >> issues with false flags for loop interdependencies.  Within the parfor
> >> body, I am performing left-indexing over unique ranges to store results
> in
> >> a variable `out`, and the engine currently expresses concerns regarding
> >> this variable.  The range bounds are short expressions, and I have
> printed
> >> out the ranges to be absolutely sure that they are unique.  If I use
> >> `check=0`, then the engine dies with a null pointer exception.
> >>
> >> What is the current strategy used for determining inter-loop
> dependencies
> >> for parfor statements?  How can we improve this?
> >>
> >> -Mike
> >>
> >> --
> >>
> >> Mike Dusenberry
> >> GitHub: github.com/dusenberrymw
> >> LinkedIn: linkedin.com/in/mikedusenberry
> >>
> >> Sent from my iPhone.
> >>
> >>
>

--001a114db3aa12bb060552206f23--