systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brendan Dwyer" <>
Subject Re: [DISCUSS] R-Interface to SystemML
Date Thu, 21 Sep 2017 23:36:48 GMT
Sorry for not responding sooner. I had some issues with my email client.


I will do my best to address as many as the points that have been raised as I can. Hopefully
Alok will be able to jump in as well once he resolves his email issues.


- I would be happy to help maintain R4ML in SystemML and I’m sure Alok would too.

- R4ML does allow arbitrary DML script to be executed via the `sysml.execute()` function.

- I think we would like to merge the entire R4ML github repository into SystemML. We could
do this the same way SparkR was merged into Spark (

- Currently the code is not ready to be merged into SystemML because we are still on the old
ML context. We have a PR in the works that will update to the newest ML context. Once that
happens we won’t need to duplicate the DML scripts.

- Documentation is generated automatically with the R package “roxygen”. We would need
to discuss how to incorporate this into the SystemML documentation. Perhaps we could look
to Spark/SparkR for ideas.

- Test are done using  the R testthat package. I can work with Alan to get that integrated
into the systemml Jenkins  server

Sent from IBM Verse

   Matthias Boehm --- Re: [DISCUSS] R-Interface to SystemML --- 
    From:"Matthias Boehm" <>, deron@apache.orgDate:Thu,
Sep 21, 2017 4:13 PMSubject:Re: [DISCUSS] R-Interface to SystemML
    I pretty much agree with Niketan and Deron. In general, it would be usefulto provide an
R API as well. However, I'm a bit concerned for two reasons:* Looking over the github repo,
apparently R4ML is not under activedevelopment/maintenance anymore (last commit Jul 20). So
who would bewilling to maintain and extend it?* Providing wrappers for our algorithm scripts
would be just a startbecause it hides our core value proposition of custom large-scale ML.Hence,
we would also need an MLContext equivalent that allows to executearbitrary DML scripts or
R functions. Is there already a tentative designof such an API and if not, who would like
to take it over?Regards,MatthiasOn Thu, Sep 21, 2017 at 3:43 PM, Deron Eriksson <>wrote:>
I agree with Niketan. An R interface definitely makes sense for SystemML.> DML itself is
based on R, so it's surprising we have Java/Scala/Python> interfaces to SystemML but we
don't have an R interface.>> Perhaps R4ML committers could supply a little more info?
For instance:> 1) Would they like to merge R4ML code into the main SystemML project>
itself? (Currently we have no modules.)> 2) What would they like to merge?> 3) If so,
how do they propose to do so?> 4) Who will do the majority of the work to add R4ML code
to SystemML? Or> who would like to volunteer to do this?> 5) Who will maintain the contributed
code? Or who would like to volunteer> to do this?> 6) Documentation is needed (fit in
SystemML documentation framework).> 7) Testing is needed (fit into SystemML testing framework).>
8) How is this packaged?>> From a technology standpoint, I think an R interface totally
makes sense.> As for a minor criticism (which I apply to other parts of SystemML too),
I> see script wrappers at
.> This tightly binds the existing DML scripts to R, which means DML> input/output modifications
could potentially require modifications to R> code.>> Deron>>>> On Thu,
Sep 21, 2017 at 11:00 AM, Niketan Pansare <>> wrote:>> >
Janardhan: I believe this is the R4ML repo:
> > r4ml . Arvind: please correct me if I am wrong.> >> > Overall, having
a R interface for SystemML is an awesome idea. Since I am> > not an R4ML expert, may
be R4ML committers can comment on how they> envision> > "two code streams to work
together".> >> > Also, comparing the features of R4ML with that of our Python
APIs will be> > useful as it might make a stronger case for R4ML.> >> >
As an FYI, here are different ways Python users can use SystemML:> > - Using MLContext
to invoke DML script (
> > systemml/beginners-guide-python#invoking-dmlpydml-> scripts-using-mlcontext>
> and
> > programming-guide.html)> > - Python algorithms wrappers (
> > systemml/beginners-guide-python#invoke-systemmls-algorithms)> > - (not important
for R4ML discussion): Python DSL (> >
> python#matrix-operations)> >> > Thanks,> >> > Niketan Pansare>
> IBM Almaden Research Center> > E-mail: npansar At> >>
>> > [image: Inactive hide details for Janardhan ---09/21/2017 04:44:02> AM---Hi>
> Arvind, This is a great idea. One question: the R4ML generat]Janardhan> > ---09/21/2017
04:44:02 AM---Hi Arvind, This is a great idea. One> question:> > the R4ML generates
any plan like the SystemML with `D> >> > From: Janardhan <>>
> To: Arvind Surve <>, "" <>
>>> > Date: 09/21/2017 04:44 AM> > Subject: Re:
[DISCUSS] R-Interface to SystemML> > ------------------------------> >> >>
>> > Hi Arvind,> >> > This is a great idea. One question: the R4ML generates
any plan like the> > SystemML with `DML` Or with providing some interface we leverage
this> > feature ?. Community effort in the sense of collaborative algorithm> >
implementation.(?)> >> > Is this the Spark-R repo (
.> > com/v2/url?u=https-3A__github.com_rstudio_sparklyr&d=DwIGaQ&> > c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_>
> 6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-> PSwyvQzq5gwX1GL0&s=6VRs_>
> J7zXj9jZEavEP8iNvVfISAjDJeM8wFL2sBnb0g&e=  ) ?> >> > Thanks,> >
Janardhan> >> > Sent with [ProtonMail](>
> url?> > siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=>
> uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=khkGV3oXz1W5m_> > ueQRuKWlNMVOXXCVhV_ytNCINjJWY&e=
) Secure Email.> >> > > -------- Original Message --------> > > Subject:
[DISCUSS] R-Interface to SystemML> > > Local Time: September 20, 2017 12:50 PM>
> > UTC Time: September 20, 2017 4:50 PM> > > From:>
> > To: <>> > >> >
> Hi,> > > R4ML is an open source project which provides a R interface to>
> SystemML.Its a bridge between SystemML and Spark-R.> > > Lets discuss here if
and how we can get two code streams work together> > to benefit development/community
effort.> > >> > > Arvind Surve | Spark Technology Center  |>
.> > com/v2/url?u=http-3A__www.spark.tc_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=>
> HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P-> 4VuICwg6yatnAEX5JBdZ->
> PSwyvQzq5gwX1GL0&s=Yj8qfo7sjGBxX547UMGfLHRZFUxCAjZDTHWe8B7jLxI&e=> >>
>> >>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message