systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fei Hu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SYSTEMML-1782) Extend parfor block indexing analysis
Date Thu, 31 Aug 2017 19:00:07 GMT

    [ https://issues.apache.org/jira/browse/SYSTEMML-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149439#comment-16149439
] 

Fei Hu edited comment on SYSTEMML-1782 at 8/31/17 6:59 PM:
-----------------------------------------------------------

[~mboehm7] and [~dusenberrymw] I'm working on this issue. The result of {{min}} function can
be computed using {{org.apache.sysml.hops.rewrite.RewriteConstantFolding}}, but the problem
is that the value of DataOps can not be derived, for example: in {{beg = ((j-1) * batch_size)
%% nrow(X_group_batch) + 1}}, {{j}} is the {{TRANSIENTREAD}} DataOps, so we could not get
the exact number for {{beg}}. Do you have ideas for this problem?
 


was (Author: tenma):
[~mboehm7] and [~dusenberrymw] I'm working on this issue. the result of {{min}} function can
be computed using {{org.apache.sysml.hops.rewrite.RewriteConstantFolding}}, but the problem
is that the value of DataOps can not be derived, for example: {{beg = ((j-1) * batch_size)
%% nrow(X_group_batch) + 1}} where {{j}} is the {{TRANSIENTREAD }} DataOps. Do you have ideas
for this problem?
 

> Extend parfor block indexing analysis
> -------------------------------------
>
>                 Key: SYSTEMML-1782
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1782
>             Project: SystemML
>          Issue Type: Task
>            Reporter: Matthias Boehm
>            Assignee: Fei Hu
>
> This task aims to better support arbitrary block indexing in a parfor context, for expressions
as this example from SYSTEMML-1774:
> {code}
>  parfor (j in 1:parallel_batches, log=DEBUG) {
>         # Get a mini-batch in this group
>         beg = ((j-1) * batch_size) %% nrow(X_group_batch) + 1
>         end = min(nrow(X_group_batch), beg + batch_size - 1)
>         X_batch = X_group_batch[beg:end,]
>         y_batch = y_group_batch[beg:end,]
> {code}
> Currently, we're unable to determine a constant size for X_batch and y_batch. However,
since the parfor body is anyway recompiled on parfor entry, we can improve upon this as follows:
> * Leverage known constants and index ranges to perform a limit analysis for linear functions
in indexing expressions. For example, the above expression for beg and end would simplify
to {{(j-1) * 32 + 1}} and  {{beg + batch_size -1}}.
> * Better handling of linear functions in right indexing HOPs - this would allow us to
determine batch_size as the number of rows in the output.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message