spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Sasaki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-7146) Should ML sharedParams be a public API?
Date Mon, 26 Oct 2015 05:24:28 GMT

    [ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973745#comment-14973745
] 

Kai Sasaki commented on SPARK-7146:
-----------------------------------

There are several times when I want to use internal resources(e.g. shared params, optimization)
of Spark as our own library or framework. 
The task to write these code again often cause trouble and long time development. In addition
to this, as you said, there might be the several implementations which has the same name but
different functionality. 

{quote}
Cons:
Users have to be careful since parameters can have different meanings for different algorithms.
{quote}

I think this is also true when even {{sharedParams}} is private because application developers
will implements their own params which retain almost same name with {{sharedParams}}. It becomes
confusing.

So basically it might be better to enable developers to use {{sharedParams}} inside their
own frameworks. It does not mean that making it public directly. As [~josephkb] proposed on
(b), it is good way to make it open for developers but create some restrictions.

> Should ML sharedParams be a public API?
> ---------------------------------------
>
>                 Key: SPARK-7146
>                 URL: https://issues.apache.org/jira/browse/SPARK-7146
>             Project: Spark
>          Issue Type: Brainstorming
>          Components: ML
>            Reporter: Joseph K. Bradley
>
> Discussion: Should the Param traits in sharedParams.scala be public?
> Pros:
> * Sharing the Param traits helps to encourage standardized Param names and documentation.
> Cons:
> * Users have to be careful since parameters can have different meanings for different
algorithms.
> * If the shared Params are public, then implementations could test for the traits.  It
is unclear if we want users to rely on these traits, which are somewhat experimental.
> Currently, the shared params are private.
> Proposal: Either
> (a) make the shared params private to encourage users to write specialized documentation
and value checks for parameters, or
> (b) design a better way to encourage overriding documentation and parameter value checks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message