systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <mboe...@googlemail.com>
Subject Re: Parfor semantics
Date Tue, 22 Nov 2016 01:07:09 GMT
yes, this came up several times - initially we only supported opt=NONE 
where users had to specify all other parameters. Meanwhile, there is a 
so-called "constrained optimizer" that does the same as the rule-based 
optimizer but respects any given parameters. Please try something like this:

parfor (i in 1:10, opt=CONSTRAINED, par=10, mode=REMOTE_SPARK) {
      // some code here
}


Regards,
Matthias

On 11/22/2016 12:33 AM, fschueler@posteo.de wrote:
> While debugging some ParFor code it became clear that the parameters for
> parfor can be easily overwritten by the optimizer.
> One example is when I write:
>
> ```
> parfor (i in 1:10, par=10, mode=REMOTE_SPARK) {
>     // some code here
> }
> ```
>
> Depending on the data size and cluster resources, the optimizer
> (OptimizerRuleBased.java, line 844) will recognize that the work can be
> done locally and overwrite it to local execution. This might be valid
> and definitely works (in my case) but kind of contradicts what I want
> SystemML to do.
> I wonder if we should disable this optimization in case a concrete
> execution mode is given and go with the mode that is provided.
>
> Felix
>
>

Mime
View raw message