Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <1ffe92ae6a28438a3c376e207e03a69b@posteo.de>
Subject: Re: Proof of Concept: Embedded Scala DSL
To: dev@systemml.incubator.apache.org
From: "Matthias Boehm" <mboehm@us.ibm.com>
Date: Sat, 24 Sep 2016 01:11:11 -0700
References: <1ffe92ae6a28438a3c376e207e03a69b@posteo.de>
Content-type: multipart/related;
	Boundary="0__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B"
Message-Id: <OFFDCD08CE.3D1BCF6F-ON00258038.002983CB-88258038.002CF81E@notes.na.collabserv.com>
archived-at: Sat, 24 Sep 2016 08:11:32 -0000

--0__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B
Content-type: multipart/alternative;
	Boundary="1__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B"


--1__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B
Content-Transfer-Encoding: quoted-printable
Content-type: text/plain; charset=US-ASCII


thanks for sharing the summary - this is very nice. While looking over the
example, I had the following questions:

1) Output handling: It would be great to see an example how the results of
Algorithm.execute() are consumed. Do you intend to hand out our binary
matrix representation or MLContext's Matrix from which the user then
requests specific output formats? Also if there are multiple Algorithm
instances, how is the MLContext (with its internal state of lazily
evaluated intermediates) reused?

2) Scala-breeze prototyping: How do you intend to support operations that
are not supported in breeze? Examples are removeEmpty, table, aggregate,
rowIndexMax, quantile/centralmoment, cummin/cummax, and DNN operations?

3) Frame data type and operations: Do you also intend to add a frame type
and its operations? I think for this initial prototype it is not
necessarily required but please make the scope explicit.

Regards,
Matthias


From:	fschueler@posteo.de
To:	dev@systemml.incubator.apache.org
Date:	09/23/2016 04:36 PM
Subject:	Proof of Concept: Embedded Scala DSL


As discussed in the related Jira (SYSTEMML-451) I have started to
implement a prototype/proof of concept for an embedded DSL in Scala.

I have summarized the current approach in a short document that you can
find on github together with the code:
https://github.com/fschueler/emma/blob/sysml-dsl/emma-sysml-dsl/README.md
Please note that current development happens in the Emma project but
will move to an independent module in the SystemML project once the
necessary additions to Emma are merged. By having the DSL in a separate
module, we can include Scala and Emma dependencies only for the users
that actually want to use the Scala DSL.

The current code serves as a proof of concept to discuss further
development with the SystemML community. I especially welcome input from
SystemML Scala users on the usability of the API design.
Next steps will include the translation from Scala code to DML with
support of all features currently supported in DML, including control
flow structures.
Also, a coherent way of executing the generated scripts from Scala and
the interaction with outside data formats (such as Spark Dataframes)
will be integrated.

I am happy to answer your questions and discuss the described approach
here!

Felix


--1__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B
Content-Transfer-Encoding: quoted-printable
Content-type: text/html; charset=US-ASCII
Content-Disposition: inline

<html><body><p>thanks for sharing the summary - this is very nice. While lo=
oking over the example, I had the following questions:<br><br>1) Output han=
dling: It would be great to see an example how the results of Algorithm.exe=
cute() are consumed. Do you intend to hand out our binary matrix representa=
tion or MLContext's Matrix from which the user then requests specific outpu=
t formats? Also if there are multiple Algorithm instances, how is the MLCon=
text (with its internal state of lazily evaluated intermediates) reused?<br=
><br>2) Scala-breeze prototyping: How do you intend to support operations t=
hat are not supported in breeze? Examples are removeEmpty, table, aggregate=
, rowIndexMax, quantile/centralmoment, cummin/cummax, and DNN operations?<b=
r><br>3) Frame data type and operations: Do you also intend to add a frame =
type and its operations? I think for this initial prototype it is not neces=
sarily required but please make the scope explicit.<br><br>Regards,<br>Matt=
hias <br><br><br><img width=3D"16" height=3D"16" src=3D"cid:1=5F=5F=3D8FBB0=
AABDFBA055B8f9e8a93df938690918c8FB@" border=3D"0" alt=3D"Inactive hide deta=
ils for fschueler---09/23/2016 04:36:14 PM---As discussed in the related Ji=
ra (SYSTEMML-451) I have started t"><font color=3D"#424282">fschueler---09/=
23/2016 04:36:14 PM---As discussed in the related Jira (SYSTEMML-451) I hav=
e started to  implement a prototype/proof of co</font><br><br><font size=3D=
"2" color=3D"#5F5F5F">From:        </font><font size=3D"2">fschueler@posteo=
.de</font><br><font size=3D"2" color=3D"#5F5F5F">To:        </font><font si=
ze=3D"2">dev@systemml.incubator.apache.org</font><br><font size=3D"2" color=
=3D"#5F5F5F">Date:        </font><font size=3D"2">09/23/2016 04:36 PM</font=
><br><font size=3D"2" color=3D"#5F5F5F">Subject:        </font><font size=
=3D"2">Proof of Concept: Embedded Scala DSL</font><br><hr width=3D"100%" si=
ze=3D"2" align=3D"left" noshade style=3D"color:#8091A5; "><br><br><br><tt>A=
s discussed in the related Jira (SYSTEMML-451) I have started to <br>implem=
ent a prototype/proof of concept for an embedded DSL in Scala.<br><br>I hav=
e summarized the current approach in a short document that you can <br>find=
 on github together with the code: <br></tt><tt><a href=3D"https://github.c=
om/fschueler/emma/blob/sysml-dsl/emma-sysml-dsl/README.md">https://github.c=
om/fschueler/emma/blob/sysml-dsl/emma-sysml-dsl/README.md</a></tt><tt><br>P=
lease note that current development happens in the Emma project but <br>wil=
l move to an independent module in the SystemML project once the <br>necess=
ary additions to Emma are merged. By having the DSL in a separate <br>modul=
e, we can include Scala and Emma dependencies only for the users <br>that a=
ctually want to use the Scala DSL.<br><br>The current code serves as a proo=
f of concept to discuss further <br>development with the SystemML community=
. I especially welcome input from <br>SystemML Scala users on the usability=
 of the API design.<br>Next steps will include the translation from Scala c=
ode to DML with <br>support of all features currently supported in DML, inc=
luding control <br>flow structures.<br>Also, a coherent way of executing th=
e generated scripts from Scala and <br>the interaction with outside data fo=
rmats (such as Spark Dataframes) <br>will be integrated.<br><br>I am happy =
to answer your questions and discuss the described approach <br>here!<br><b=
r>Felix<br><br></tt><br><BR>
</body></html>

--1__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B--


--0__=8FBB0AABDFBA055B8f9e8a93df938690918c8FBB0AABDFBA055B--