systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajarshi Bhadra <bhadrarajars...@gmail.com>
Subject Parfor optimizer getting stuck
Date Mon, 04 Sep 2017 06:51:36 GMT
Hi,

I am working on a custom tree based algorithm which I am trying to develop
using SystemML. I am using Version 1.0.0-SNAPSHOT. Now my issue is the
parfor statement is getting stuck somewhere and I am not getting any error
report or output so I am unable to determine what the issue might be.
However I have been able to get a log of the parfor which is as follows


17/09/04 06:59:12 DEBUG Optimizer: --- RULEBASED OPTIMIZER -------
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: Optimize w/
max_mem=63716MB/InfinityMB/InfinityMB, max_k=32/1/1).
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: estimated mem (serial
exec) M=52MB
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set data
partitioner' - result=NONE ()
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'remove
unnecessary compare matrix' - result=false ()
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set result
partitioning' - result=false
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: estimated new mem (serial
exec) M=52MB
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: estimated new mem (serial
exec, all CP) M=273068MB
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: estimated new mem (cond
partitioning) M=52MB
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set execution
strategy' - result=REMOTE_SPARK (recompile=true
)
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set operation
exec type CP' - result=198
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'enable data
colocation' - result=false
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set partition
replication factor' - result=false
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set export
replication factor' - result=true (3)
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'enable nested
parallelism' - result=false
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set degree of
parallelism' - result=(see EXPLAIN)
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set task
partitioner' - result=NAIVE
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set fused data
partitioning and execution' - result=false
17/09/04 06:59:12 DEBUG Optimizer: RULEBASED OPT: rewrite 'set transpose
sparse vector operations' - result=false

It would be great if someone can help me out with this issue

Thank you

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message