spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yli...@apache.org
Subject spark git commit: [SPARK-21087][ML][FOLLOWUP] Sync SharedParamsCodeGen and sharedParams.
Date Wed, 13 Dec 2017 01:37:05 GMT
Repository: spark
Updated Branches:
  refs/heads/master 17cdabb88 -> b03af8b58


[SPARK-21087][ML][FOLLOWUP] Sync SharedParamsCodeGen and sharedParams.

## What changes were proposed in this pull request?
#19208 modified ```sharedParams.scala```, but didn't generated by ```SharedParamsCodeGen.scala```.
This involves mismatch between them.

## How was this patch tested?
Existing test.

Author: Yanbo Liang <ybliang8@gmail.com>

Closes #19958 from yanboliang/spark-21087.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b03af8b5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b03af8b5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b03af8b5

Branch: refs/heads/master
Commit: b03af8b582b9b71b09eaf3a1c01d1b3ef5f072e8
Parents: 17cdabb
Author: Yanbo Liang <ybliang8@gmail.com>
Authored: Tue Dec 12 17:37:01 2017 -0800
Committer: Yanbo Liang <ybliang8@gmail.com>
Committed: Tue Dec 12 17:37:01 2017 -0800

----------------------------------------------------------------------
 .../spark/ml/param/shared/SharedParamsCodeGen.scala       |  8 ++++----
 .../org/apache/spark/ml/param/shared/sharedParams.scala   | 10 ++++++----
 2 files changed, 10 insertions(+), 8 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/b03af8b5/mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
----------------------------------------------------------------------
diff --git a/mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
b/mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
index c540629..a267bbc 100644
--- a/mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
+++ b/mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
@@ -84,10 +84,10 @@ private[shared] object SharedParamsCodeGen {
       ParamDesc[String]("solver", "the solver algorithm for optimization", finalFields =
false),
       ParamDesc[Int]("aggregationDepth", "suggested depth for treeAggregate (>= 2)", Some("2"),
         isValid = "ParamValidators.gtEq(2)", isExpertParam = true),
-      ParamDesc[Boolean]("collectSubModels", "If set to false, then only the single best
" +
-        "sub-model will be available after fitting. If set to true, then all sub-models will
be " +
-        "available. Warning: For large models, collecting all sub-models can cause OOMs on
the " +
-        "Spark driver.",
+      ParamDesc[Boolean]("collectSubModels", "whether to collect a list of sub-models trained
" +
+        "during tuning. If set to false, then only the single best sub-model will be available
" +
+        "after fitting. If set to true, then all sub-models will be available. Warning: For
" +
+        "large models, collecting all sub-models can cause OOMs on the Spark driver",
         Some("false"), isExpertParam = true)
     )
 

http://git-wip-us.apache.org/repos/asf/spark/blob/b03af8b5/mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala
----------------------------------------------------------------------
diff --git a/mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala b/mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala
index 34aa38a..0004f08 100644
--- a/mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala
+++ b/mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala
@@ -470,15 +470,17 @@ trait HasAggregationDepth extends Params {
 }
 
 /**
- * Trait for shared param collectSubModels (default: false).
+ * Trait for shared param collectSubModels (default: false). This trait may be changed or
+ * removed between minor versions.
  */
-private[ml] trait HasCollectSubModels extends Params {
+@DeveloperApi
+trait HasCollectSubModels extends Params {
 
   /**
-   * Param for whether to collect a list of sub-models trained during tuning.
+   * Param for whether to collect a list of sub-models trained during tuning. If set to false,
then only the single best sub-model will be available after fitting. If set to true, then
all sub-models will be available. Warning: For large models, collecting all sub-models can
cause OOMs on the Spark driver.
    * @group expertParam
    */
-  final val collectSubModels: BooleanParam = new BooleanParam(this, "collectSubModels", "whether
to collect a list of sub-models trained during tuning")
+  final val collectSubModels: BooleanParam = new BooleanParam(this, "collectSubModels", "whether
to collect a list of sub-models trained during tuning. If set to false, then only the single
best sub-model will be available after fitting. If set to true, then all sub-models will be
available. Warning: For large models, collecting all sub-models can cause OOMs on the Spark
driver")
 
   setDefault(collectSubModels, false)
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message