Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 1 Jun 2016 17:16:59 +0000 (UTC)
From: "Yanbo Liang (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.12973335.1464336400000.4304.1464801419171@Atlassian.JIRA>
In-Reply-To: <JIRA.12973335.1464336400000@Atlassian.JIRA>
References: <JIRA.12973335.1464336400000@Atlassian.JIRA> <JIRA.12973335.1464336400587@arcas>
Subject: [jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 01 Jun 2016 17:17:00 -0000


     [ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yanbo Liang updated SPARK-15605:
--------------------------------
    Description: 
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html .


  was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html .

To fix the issue mentioned in the mailing list, we can implement maxIter as following:
{code}
  private IntParam maxIter_;

  public IntParam maxIter() {
    return maxIter_;
  }

  public int getMaxIter() {
    return (Integer) getOrDefault(maxIter_);
  }

  public MyJavaLogisticRegression setMaxIter(int value) {
    return (MyJavaLogisticRegression) set(maxIter_, value);
  }

  private void init() {
    maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
    setDefault(maxIter(), 100);
  }
{code}

But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala Params.

For example, in Scala we have: 
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}

Otherwise, "Param.parent" will be null for each param and it will throw exceptions when calling "Param.hasParam()".

I think the Java compatibility for Params/Param should be further defined in the next release cycle, and it's better to remove the "JavaDeveloperApiExample" at present. Since currently we can not well support users to implement their own algorithm using Estimator/Transformer by Java, it may mislead users.


> ML JavaDeveloperApiExample is broken
> ------------------------------------
>
>                 Key: SPARK-15605
>                 URL: https://issues.apache.org/jira/browse/SPARK-15605
>             Project: Spark
>          Issue Type: Bug
>          Components: Examples, ML
>    Affects Versions: 2.0.0
>            Reporter: Yanbo Liang
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html .


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org