systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arvind Surve (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SYSTEMML-930) Warn user to Set MaxResultSize to unlimited in MLContext based call
Date Mon, 19 Sep 2016 04:33:20 GMT

    [ https://issues.apache.org/jira/browse/SYSTEMML-930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502250#comment-15502250
] 

Arvind Surve commented on SYSTEMML-930:
---------------------------------------

Setting MaxResultSize after SparkContext started does not get reflected. This to be set before
starting SparkContext. So if user has not set earlier, warn user for the same.

> Warn user to Set MaxResultSize to unlimited in MLContext based call
> -------------------------------------------------------------------
>
>                 Key: SYSTEMML-930
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-930
>             Project: SystemML
>          Issue Type: Improvement
>            Reporter: Arvind Surve
>            Assignee: Arvind Surve
>
> Currently MaxResultSize has not set in the case MlContext call being made. In case user
does not specify it through spark.driver.maxResultSize, it gets set to default size (1g).
In order to get unlimited data to driver we have set this size to unlimited in other calls
like DML script calls through spark-submit. We need to do similar for MlContext based call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message