spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weichen Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs
Date Thu, 20 Sep 2018 11:39:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621862#comment-16621862
] 

Weichen Xu commented on SPARK-25321:
------------------------------------

[~mengxr]  mleap is NOT compatible with the tree Node breaking changes. compile failed:
```
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/ops/classification/DecisionTreeClassifierOp.scala:34:
type mismatch;
[error]  found   : org.apache.spark.ml.tree.Node
[error]  required: org.apache.spark.ml.tree.ClassificationNode
[error]         rootNode = rootNode,
[error]                    ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/ops/clustering/LDAModelOp.scala:59:
reassignment to val
[error]       oldLocalModel = oldLocalModel,
[error]                     ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/ops/regression/DecisionTreeRegressionOp.scala:33:
type mismatch;
[error]  found   : org.apache.spark.ml.tree.Node
[error]  required: org.apache.spark.ml.tree.RegressionNode
[error]         rootNode = rootNode,
[error]                    ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:50:
trait LeafNode is abstract; cannot be instantiated
[error]     new tree.LeafNode(prediction = node.values.indexOf(node.values.max),
[error]     ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:50:
not found: value prediction
[error]     new tree.LeafNode(prediction = node.values.indexOf(node.values.max),
[error]                       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:51:
not found: value impurity
[error]       impurity = 0.0,
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:52:
not found: value impurityStats
[error]       impurityStats = calc)
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:75:
trait InternalNode is abstract; cannot be instantiated
[error]     new tree.InternalNode(split = split,
[error]     ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:75:
reassignment to val
[error]     new tree.InternalNode(split = split,
[error]                                 ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:76:
not found: value leftChild
[error]       leftChild = left,
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:77:
not found: value rightChild
[error]       rightChild = right,
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:78:
not found: value prediction
[error]       prediction = 0.0,
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:79:
not found: value gain
[error]       gain = 0.0,
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:80:
not found: value impurity
[error]       impurity = 0.0,
[error]       ^
[error] /Users/weichenxu/work/projects/mySpark/mleap/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/tree/decision/SparkNodeWrapper.scala:81:
not found: value impurityStats
[error]       impurityStats = null)
[error]       ^
[error] 15 errors found
```

> ML, Graph 2.4 QA: API: New Scala APIs, docs
> -------------------------------------------
>
>                 Key: SPARK-25321
>                 URL: https://issues.apache.org/jira/browse/SPARK-25321
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Documentation, GraphX, ML, MLlib
>    Affects Versions: 2.4.0
>            Reporter: Weichen Xu
>            Assignee: Yanbo Liang
>            Priority: Blocker
>
> Audit new public Scala APIs added to MLlib & GraphX. Take note of:
>  * Protected/public classes or methods. If access can be more private, then it should
be.
>  * Also look for non-sealed traits.
>  * Documentation: Missing? Bad links or formatting?
> *Make sure to check the object doc!*
> As you find issues, please create JIRAs and link them to this issue. 
> For *user guide issues* link the new JIRAs to the relevant user guide QA issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message