systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dusenberr...@gmail.com
Subject Re: Evaluate a scalar DAG during compilation
Date Mon, 24 Apr 2017 22:47:59 GMT
Okay, interesting.  I have a few scenarios for which this is not triggered.  I'll open a new
JIRA issue to share the scenarios so that we can fix the issue.

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Apr 24, 2017, at 2:56 PM, Matthias Boehm <mboehm7@googlemail.com> wrote:
> 
> yes, we already do constant folding - the details are in
> org.apache.sysml.hops.rewrite.RewriteConstantFolding
> 
> In order to ensure consistency with our runtime, we actually generate
> instructions for these sub dags, execute them and finally replace the dag
> with the computed literal.
> 
> Regards,
> Matthias
> 
>> On Mon, Apr 24, 2017 at 2:45 PM, <dusenberrymw@gmail.com> wrote:
>> 
>> During compilation, is it possible to evaluate a scalar sub-DAG of scalar
>> operations in which all leaf nodes are literals to allow for replacement
>> with a literal?  For example, in our `nn` library, our convolution and
>> pooling layers have to pass around the spatial dimensions (height and
>> width) of the images that are stretched out into rows of the input/output
>> matrices.  These output dimensions are computed within the forward
>> functions of the above layers as small scalar equations.  From a
>> mathematical standpoint, these sizes can be determined at compile time, and
>> it is nice to have these size equations in DML (v.s. hiding them inside the
>> engine within built-in functions).  However, we do not currently evaluate
>> these expressions during compilation, and thus we are left with unknown
>> sizes even during recompilation.  This naturally leads to max memory
>> estimates and thus often leads to unnecessary distributed runtime ops
>> rather than simple CP ones.
>> 
>> Thoughts?
>> 
>> 
>> -Mike
>> 
>> --
>> 
>> Mike Dusenberry
>> GitHub: github.com/dusenberrymw
>> LinkedIn: linkedin.com/in/mikedusenberry
>> 
>> Sent from my iPhone.
>> 
>> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message