spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yuhao yang (JIRA)" <>
Subject [jira] [Commented] (SPARK-7514) Add MinMaxScaler to feature transformation
Date Mon, 11 May 2015 05:41:00 GMT


yuhao yang commented on SPARK-7514:

Class name has always been MinMaxScaler in the code, yet I named jira wrongly...

For the parameters, currently the model looks like:
class MinMaxScalerModel (
+    val min: Vector,
+    val max: Vector,
+    var newBase: Double,
+    var scale: Double) extends VectorTransformer 

I have used min, max to store the model statistics. In some articles, the range bounds are
named newMin / newMax (I think it can be confusing). 
ran out of variable names here...

setCenterScale looks good.

> Add MinMaxScaler to feature transformation
> ------------------------------------------
>                 Key: SPARK-7514
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: MLlib
>            Reporter: yuhao yang
>   Original Estimate: 24h
>  Remaining Estimate: 24h
> Add a popular scaling method to feature component, which is commonly known as min-max
normalization or Rescaling.
> Core function is,
> Normalized( x ) = (x - min) / (max - min) * scale + newBase
> where newBase and scale are parameters of the VectorTransformer. newBase is the new minimum
number for the feature, and scale controls the range after transformation. This is a little
complicated than the basic MinMax normalization, yet it provides flexibility so that users
can control the range more specifically. like [0.1, 0.9] in some NN application.
> for case that max == min, 0.5 is used as the raw value.
> reference:

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message