systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <mboe...@googlemail.com>
Subject Re: [DISCUSS] R-Interface to SystemML
Date Thu, 21 Sep 2017 23:13:09 GMT
I pretty much agree with Niketan and Deron. In general, it would be useful
to provide an R API as well. However, I'm a bit concerned for two reasons:

* Looking over the github repo, apparently R4ML is not under active
development/maintenance anymore (last commit Jul 20). So who would be
willing to maintain and extend it?

* Providing wrappers for our algorithm scripts would be just a start
because it hides our core value proposition of custom large-scale ML.
Hence, we would also need an MLContext equivalent that allows to execute
arbitrary DML scripts or R functions. Is there already a tentative design
of such an API and if not, who would like to take it over?

Regards,
Matthias


On Thu, Sep 21, 2017 at 3:43 PM, Deron Eriksson <deroneriksson@gmail.com>
wrote:

> I agree with Niketan. An R interface definitely makes sense for SystemML.
> DML itself is based on R, so it's surprising we have Java/Scala/Python
> interfaces to SystemML but we don't have an R interface.
>
> Perhaps R4ML committers could supply a little more info? For instance:
> 1) Would they like to merge R4ML code into the main SystemML project
> itself? (Currently we have no modules.)
> 2) What would they like to merge?
> 3) If so, how do they propose to do so?
> 4) Who will do the majority of the work to add R4ML code to SystemML? Or
> who would like to volunteer to do this?
> 5) Who will maintain the contributed code? Or who would like to volunteer
> to do this?
> 6) Documentation is needed (fit in SystemML documentation framework).
> 7) Testing is needed (fit into SystemML testing framework).
> 8) How is this packaged?
>
> From a technology standpoint, I think an R interface totally makes sense.
> As for a minor criticism (which I apply to other parts of SystemML too), I
> see script wrappers at https://github.com/SparkTC/r4ml/tree/master/R4ML/R.
> This tightly binds the existing DML scripts to R, which means DML
> input/output modifications could potentially require modifications to R
> code.
>
> Deron
>
>
>
> On Thu, Sep 21, 2017 at 11:00 AM, Niketan Pansare <npansar@us.ibm.com>
> wrote:
>
> > Janardhan: I believe this is the R4ML repo: https://github.com/SparkTC/
> > r4ml . Arvind: please correct me if I am wrong.
> >
> > Overall, having a R interface for SystemML is an awesome idea. Since I am
> > not an R4ML expert, may be R4ML committers can comment on how they
> envision
> > "two code streams to work together".
> >
> > Also, comparing the features of R4ML with that of our Python APIs will be
> > useful as it might make a stronger case for R4ML.
> >
> > As an FYI, here are different ways Python users can use SystemML:
> > - Using MLContext to invoke DML script (http://apache.github.io/
> > systemml/beginners-guide-python#invoking-dmlpydml-
> scripts-using-mlcontext
> > and http://apache.github.io/systemml/spark-mlcontext-
> > programming-guide.html)
> > - Python algorithms wrappers (http://apache.github.io/
> > systemml/beginners-guide-python#invoke-systemmls-algorithms)
> > - (not important for R4ML discussion): Python DSL (
> > http://apache.github.io/systemml/beginners-guide-
> python#matrix-operations)
> >
> > Thanks,
> >
> > Niketan Pansare
> > IBM Almaden Research Center
> > E-mail: npansar At us.ibm.com
> > http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
> >
> > [image: Inactive hide details for Janardhan ---09/21/2017 04:44:02
> AM---Hi
> > Arvind, This is a great idea. One question: the R4ML generat]Janardhan
> > ---09/21/2017 04:44:02 AM---Hi Arvind, This is a great idea. One
> question:
> > the R4ML generates any plan like the SystemML with `D
> >
> > From: Janardhan <j143@protonmail.com>
> > To: Arvind Surve <acs_s@yahoo.com.INVALID>, "dev@systemml.apache.org" <
> > dev@systemml.apache.org>
> > Date: 09/21/2017 04:44 AM
> > Subject: Re: [DISCUSS] R-Interface to SystemML
> > ------------------------------
> >
> >
> >
> > Hi Arvind,
> >
> > This is a great idea. One question: the R4ML generates any plan like the
> > SystemML with `DML` Or with providing some interface we leverage this
> > feature ?. Community effort in the sense of collaborative algorithm
> > implementation.(?)
> >
> > Is this the Spark-R repo ( https://urldefense.proofpoint.
> > com/v2/url?u=https-3A__github.com_rstudio_sparklyr&d=DwIGaQ&
> > c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_
> > 6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-
> PSwyvQzq5gwX1GL0&s=6VRs_
> > J7zXj9jZEavEP8iNvVfISAjDJeM8wFL2sBnb0g&e=  ) ?
> >
> > Thanks,
> > Janardhan
> >
> > Sent with [ProtonMail](https://urldefense.proofpoint.com/v2/
> > url?u=https-3A__protonmail.com&d=DwIGaQ&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=
> > uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=khkGV3oXz1W5m_
> > ueQRuKWlNMVOXXCVhV_ytNCINjJWY&e= ) Secure Email.
> >
> > > -------- Original Message --------
> > > Subject: [DISCUSS] R-Interface to SystemML
> > > Local Time: September 20, 2017 12:50 PM
> > > UTC Time: September 20, 2017 4:50 PM
> > > From: acs_s@yahoo.com.INVALID
> > > To: dev@systemml.apache.org <dev@systemml.apache.org>
> > >
> > > Hi,
> > > R4ML is an open source project which provides a R interface to
> > SystemML.Its a bridge between SystemML and Spark-R.
> > > Lets discuss here if and how we can get two code streams work together
> > to benefit development/community effort.
> > >
> > > Arvind Surve | Spark Technology Center  |
> https://urldefense.proofpoint.
> > com/v2/url?u=http-3A__www.spark.tc_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=
> > HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P-
> 4VuICwg6yatnAEX5JBdZ-
> > PSwyvQzq5gwX1GL0&s=Yj8qfo7sjGBxX547UMGfLHRZFUxCAjZDTHWe8B7jLxI&e=
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message