spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yanbo Liang (JIRA)" <>
Subject [jira] [Resolved] (SPARK-18710) Add offset to GeneralizedLinearRegression models
Date Fri, 30 Jun 2017 12:04:01 GMT


Yanbo Liang resolved SPARK-18710.
          Resolution: Fixed
       Fix Version/s: 2.3.0
    Target Version/s: 2.3.0

> Add offset to GeneralizedLinearRegression models
> ------------------------------------------------
>                 Key: SPARK-18710
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: ML
>    Affects Versions: 2.0.2
>            Reporter: Wayne Zhang
>            Assignee: Wayne Zhang
>              Labels: features
>             Fix For: 2.3.0
>   Original Estimate: 10h
>  Remaining Estimate: 10h
> The current GeneralizedLinearRegression model does not support offset. The offset can
be useful to take into account exposure, or for testing incremental effect of new variables.
It is possible to use weights in current environment to achieve the same effect of specifying
offset for certain models, e.g., Poisson & Binomial with log offset, it is desirable to
have the offset option to work with more general cases, e.g., negative offset or offset that
is hard to specify using weights (e.g., offset to the probability rather than odds in logistic
> Effort would involve:
> * update regression class to support offsetCol
> * update IWLS to take into account of offset
> * add test case for offset
> I can start working on this if the community approves this feature. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message