spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-26172) Unify String Params' case-insensitivity in ML
Date Tue, 11 Dec 2018 10:58:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-26172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16716831#comment-16716831
] 

ASF GitHub Bot commented on SPARK-26172:
----------------------------------------

SparkQA commented on issue #23144: [SPARK-26172][ML][WIP] Unify String Params' case-insensitivity
in ML
URL: https://github.com/apache/spark/pull/23144#issuecomment-446160698
 
 
   **[Test build #99967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99967/testReport)**
for PR 23144 at commit [`11bd906`](https://github.com/apache/spark/commit/11bd9065a850cea6ec1678fc8883746a393e3e02).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Unify String Params' case-insensitivity in ML
> ---------------------------------------------
>
>                 Key: SPARK-26172
>                 URL: https://issues.apache.org/jira/browse/SPARK-26172
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 3.0.0
>            Reporter: zhengruifeng
>            Priority: Major
>
> For now, there are three ways to deal with case-insensitivity in ML:
> 1, support case-insensitivity, e.g. {{LogisticRegression}};
> 2, support case-insensitivity, but with getter returning the lower case value (not
the value passed to setter), e.g. {{ALS}},{{DecisionTreeClassifier}};
> 3, do not support case-insensitivity, e.g. {{NaiveBayes}}
>  
> This situation result in confusion in usage. 
> I think we should choose the *first* way to support case-insensitivity of all non-columnName
string params, including:
>  * LogisticRegression: family
>  * MultilayerPerceptronClassifier: {{solver}}
>  * NaiveBayes: modelType
>  * DecisionTreeClassifier: impurity
>  * RandomForestClassifier: featureSubsetStrategy, impurity
>  * GBTClassifier: featureSubsetStrategy, impurity, {{lossType}}
>  * {{}}
>  * LinearRegression: solver, loss
>  * GeneralizedLinearRegression: family, link, solver
>  * DecisionTreeRegressor: impurity
>  * RandomForestRegressor: featureSubsetStrategy, impurity
>  * GBTRegressor: featureSubsetStrategy, impurity, {{lossType}}
>  * {{}}
>  * {\{KMeans: }}initMode
>  * LDA: optimizer
>  * PowerIterationClustering\{{: }}initMode
>  * 
>  * ALS: coldStartStrategy, intermediateStorageLevel, finalStorageLevel
>  * 
>  * Bucketizer: handleInvalid
>  * ChiSqSelector: selectorType
>  * Imputer: strategy
>  * QuantileDiscretizer: handleInvalid
>  * RFormula: handleInvalid, stringIndexerOrderType
>  * StringIndexer: handleInvalid, stringOrderType
>  * VectorAssembler: handleInvalid
>  * VectorIndexer: handleInvalid
>  * VectorSizeHint: handleInvalid
>  * OneHotEncoderEstimator: handleInvalid (*this will be let alone until the breaking
change*)
>  * 
>  * BinaryClassificationEvaluator: metricName
>  * MulticlassClassificationEvaluator: metricName
>  * RegressionEvaluator: metricName
>  * ClusteringEvaluator: metricName, distanceMeasure
>  
>  
>  
> To to this:
>  * methods {{lowerCaseInArray}} and {{upperCaseInArray}} are created in {{ParamValidators}}
to check case-insensitivity;
>  * methods  {{{{$$(param: Param[String])}}}} and {{%%(param: Param[String])}} are created
in trait {{Params}} to lower/upper the param value conveniently, and this can minimize the
modifications in existing codes, since in many cases we only need to change {{$(param)}}
to {{$$\{param}}};
>  * in *SharedParamsCodeGen*, *handleInvalid* and *{{distanceMeasure}}* are updated to
use  lowerCaseInArray
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message