systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm (JIRA)" <>
Subject [jira] [Created] (SYSTEMML-2120) Improve codegen optimizer (pruning effectiveness)
Date Sat, 03 Feb 2018 08:41:00 GMT
Matthias Boehm created SYSTEMML-2120:

             Summary: Improve codegen optimizer (pruning effectiveness)
                 Key: SYSTEMML-2120
             Project: SystemML
          Issue Type: Sub-task
            Reporter: Matthias Boehm

The codegen optimizer applies various techniques to prune the search space, including the
partitioning into independent problems, the restriction to interesting points, as well as
cost-based and structural pruning. These techniques work very well for most use case rendering
the codegen overhead negligible. However, there are still exceptions where the additional
compilation overhead leads to a slowdown. One of these examples is lenet, the below codegen
statistics are obtained from 1000 iterations over mnist60k.

HOP DAGs recompiled (PRED, SB): 0/1154.
HOP DAGs recompile time:        237.579 sec.
Codegen compile (DAG,CP,JC):    1176/17450/36.
Codegen enum (ALLt/p,EVALt/p):  1064354993445/16387318/11342971/11331811.
Codegen compile times (DAG,JC): 234.174/0.835 sec.
Codegen plan cache hits:        17408/17444.

Pruning reduced the number of plans from > 1 trillion to 11 million, which is still too
high causing an overhead of 234s for a total execution time of 425s.

This message was sent by Atlassian JIRA

View raw message