Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
Message-Id: <201601022025.u02KPAwe002085@d01av02.pok.ibm.com>
In-Reply-To: 
 <CAO1aQR_kxwTKDAwBgGLw5=Oj5fYU89y4SACpauuE6ck-Cs0fpg@mail.gmail.com>
Subject: Re: POC Eclipse IDE for DML
To: dev@systemml.incubator.apache.org
From: "Matthias Boehm" <mboehm@us.ibm.com>
Date: Sat, 2 Jan 2016 12:25:04 -0800
References: 
 <CAOcw5OxYocDFDh60iM9foco25SY=k_62hwaMR09ZihwUHBH8FA@mail.gmail.com>
 <CAO1aQR_kxwTKDAwBgGLw5=Oj5fYU89y4SACpauuE6ck-Cs0fpg@mail.gmail.com>
Content-type: multipart/related;
	Boundary="0__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F"

--0__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F
Content-type: multipart/alternative;
	Boundary="1__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F"


--1__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F
Content-Transfer-Encoding: quoted-printable
Content-type: text/plain; charset=US-ASCII


just to clarify, our parser grammar does not include builtin functions
because they are -- in contrast to keywords and language constructs -- not
part of the DML/PyDML syntax. This is important for both maintainability
and flexibility. For example, it allows you to define a variable or
user-defined function with the name 't' although there exists a builtin
function t() for transpose operations.

Regards,
Matthias


From:	Deron Eriksson <deroneriksson@gmail.com>
To:	dev@systemml.incubator.apache.org
Date:	01/01/2016 11:38 AM
Subject:	Re: POC Eclipse IDE for DML


Hi Nakul,

This is very cool! I see great value in IDE tools such as an Eclipse editor
for DML/PyDML. Highlighting syntax errors helps people catch syntax
mistakes before running the script, and IDE code completion is a great
productivity boost.

Also, I like how clean and readable the grammar is. I don't know how others
feel, but it is so clean that, if flushed out, it might be a useful
addition to the DML Language Reference. It does things such as enumerate
the built-in functions, which is handy when trying to wrap your mind around
the language. The Dml.g4 file is very powerful and useful but I have a
harder time understanding the language syntax when looking at it since
things like built-in functions aren't included in it. I wonder if there is
a way to integrate some of your work into Dml.g4?

I haven't done Xtext before. Do you have any advice to get started if I'd
like to try this out in my Eclipse (for instance, to look at the semicolon
issue)?

Deron


On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <nakul02@gmail.com> wrote:

> Hi,
>
> I've been trying to build a Proof of Concept IDE for DML using Xtext.
>
> https://github.com/nakul02/sysml.dml
>
> Grammar File :
>
>
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xt=
ext

>
> Here are some screenshots:
> http://imgur.com/a/ZJyg7
>
> The files shown are from the algorithms folder.
>
https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
>
> (There was no particular reason to have chosen these files over others)
>
> For this POC, my only goal was to get syntax highlighting to work in
> Eclipse.
> SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3.
Some
> amount of work was needed on the grammar before it could be used it for
> Xtext.
>
> From the screenshots, you can see that eclipse thinks there are syntactic
> errors in the files. This has to do with statements ending in an optional
> semicolon.
>
> I only supplied xtext a grammar file from which to generate the IDE. I
have
> not yet worked on the generated code to add any other features.
> I could not come up with a LL(*) grammar that would let me express
optional
> semicolons at the end of statements. For the time being, the grammar
> requires semicolons at the end of statements (as you can see in the
grammar
> file :
>
>
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xt=
ext

> )
>
> I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> dealing with this (assuming no way of modifying the grammar to fix this):
>
> 1. Support a subset of DML in the XText based tools
> 2. Modify the xtext generated tools to fix this
> 3. Write an eclipse plugin from scratch with complete control over lexer
> and parser (among other components)
>
> Option 1 requires the least amount of work.
> I am not sure between options 2 & 3, which would be more work.
>
>
> Is this useful to anyone?
> If so, are there any thoughts or suggestions on how to approach the
> optional semicolon problem?
>
>
> Nakul Jindal
> https://github.com/nakul02/
>


--1__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F
Content-Transfer-Encoding: quoted-printable
Content-type: text/html; charset=US-ASCII
Content-Disposition: inline

<html><body><p>just to clarify, our parser grammar does not include builtin=
 functions because they are -- in contrast to keywords and language constru=
cts -- not part of the DML/PyDML syntax. This is important for both maintai=
nability and flexibility. For example, it allows you to define a variable o=
r user-defined function with the name 't' although there exists a builtin f=
unction t() for transpose operations.  <br><br>Regards,<br>Matthias<br><br>=
<br><img width=3D"16" height=3D"16" src=3D"cid:1=5F=5F=3D8FBBF5BDDFFCB59F8f=
9e8a93df938690918c8FB@" border=3D"0" alt=3D"Inactive hide details for Deron=
 Eriksson ---01/01/2016 11:38:27 AM---Hi Nakul, This is very cool! I see gr=
eat value in IDE tool"><font color=3D"#424282">Deron Eriksson ---01/01/2016=
 11:38:27 AM---Hi Nakul, This is very cool! I see great value in IDE tools =
such as an Eclipse editor</font><br><br><font size=3D"2" color=3D"#5F5F5F">=
From:        </font><font size=3D"2">Deron Eriksson &lt;deroneriksson@gmail=
.com&gt;</font><br><font size=3D"2" color=3D"#5F5F5F">To:        </font><fo=
nt size=3D"2">dev@systemml.incubator.apache.org</font><br><font size=3D"2" =
color=3D"#5F5F5F">Date:        </font><font size=3D"2">01/01/2016 11:38 AM<=
/font><br><font size=3D"2" color=3D"#5F5F5F">Subject:        </font><font s=
ize=3D"2">Re: POC Eclipse IDE for DML</font><br><hr width=3D"100%" size=3D"=
2" align=3D"left" noshade style=3D"color:#8091A5; "><br><br><br><tt>Hi Naku=
l,<br><br>This is very cool! I see great value in IDE tools such as an Ecli=
pse editor<br>for DML/PyDML. Highlighting syntax errors helps people catch =
syntax<br>mistakes before running the script, and IDE code completion is a =
great<br>productivity boost.<br><br>Also, I like how clean and readable the=
 grammar is. I don't know how others<br>feel, but it is so clean that, if f=
lushed out, it might be a useful<br>addition to the DML Language Reference.=
 It does things such as enumerate<br>the built-in functions, which is handy=
 when trying to wrap your mind around<br>the language. The Dml.g4 file is v=
ery powerful and useful but I have a<br>harder time understanding the langu=
age syntax when looking at it since<br>things like built-in functions aren'=
t included in it. I wonder if there is<br>a way to integrate some of your w=
ork into Dml.g4?<br><br>I haven't done Xtext before. Do you have any advice=
 to get started if I'd<br>like to try this out in my Eclipse (for instance,=
 to look at the semicolon<br>issue)?<br><br>Deron<br><br><br><br>On Mon, De=
c 21, 2015 at 2:44 PM, Nakul Jindal &lt;nakul02@gmail.com&gt; wrote:<br><br=
>&gt; Hi,<br>&gt;<br>&gt; I've been trying to build a Proof of Concept IDE =
for DML using Xtext.<br>&gt;<br>&gt; </tt><tt><a href=3D"https://github.com=
/nakul02/sysml.dml">https://github.com/nakul02/sysml.dml</a></tt><tt><br>&g=
t;<br>&gt; Grammar File :<br>&gt;<br>&gt; </tt><tt><a href=3D"https://githu=
b.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext">https://=
github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext</a><=
/tt><tt><br>&gt;<br>&gt; Here are some screenshots:<br>&gt; </tt><tt><a hre=
f=3D"http://imgur.com/a/ZJyg7">http://imgur.com/a/ZJyg7</a></tt><tt><br>&gt=
;<br>&gt; The files shown are from the algorithms folder.<br>&gt; </tt><tt>=
<a href=3D"https://github.com/apache/incubator-systemml/tree/master/scripts=
/algorithms">https://github.com/apache/incubator-systemml/tree/master/scrip=
ts/algorithms</a></tt><tt><br>&gt;<br>&gt; (There was no particular reason =
to have chosen these files over others)<br>&gt;<br>&gt; For this POC, my on=
ly goal was to get syntax highlighting to work in<br>&gt; Eclipse.<br>&gt; =
SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3. Some=
<br>&gt; amount of work was needed on the grammar before it could be used i=
t for<br>&gt; Xtext.<br>&gt;<br>&gt; From the screenshots, you can see that=
 eclipse thinks there are syntactic<br>&gt; errors in the files. This has t=
o do with statements ending in an optional<br>&gt; semicolon.<br>&gt;<br>&g=
t; I only supplied xtext a grammar file from which to generate the IDE. I h=
ave<br>&gt; not yet worked on the generated code to add any other features.=
<br>&gt; I could not come up with a LL(*) grammar that would let me express=
 optional<br>&gt; semicolons at the end of statements. For the time being, =
the grammar<br>&gt; requires semicolons at the end of statements (as you ca=
n see in the grammar<br>&gt; file :<br>&gt;<br>&gt; </tt><tt><a href=3D"htt=
ps://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext=
">https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.=
xtext</a></tt><tt><br>&gt; )<br>&gt;<br>&gt; I discussed this problem with =
Fred(@frreiss) and he suggested 3 ways of<br>&gt; dealing with this (assumi=
ng no way of modifying the grammar to fix this):<br>&gt;<br>&gt; 1. Support=
 a subset of DML in the XText based tools<br>&gt; 2. Modify the xtext gener=
ated tools to fix this<br>&gt; 3. Write an eclipse plugin from scratch with=
 complete control over lexer<br>&gt; and parser (among other components)<br=
>&gt;<br>&gt; Option 1 requires the least amount of work.<br>&gt; I am not =
sure between options 2 &amp; 3, which would be more work.<br>&gt;<br>&gt;<b=
r>&gt; Is this useful to anyone?<br>&gt; If so, are there any thoughts or s=
uggestions on how to approach the<br>&gt; optional semicolon problem?<br>&g=
t;<br>&gt;<br>&gt; Nakul Jindal<br>&gt; </tt><tt><a href=3D"https://github.=
com/nakul02/">https://github.com/nakul02/</a></tt><tt><br>&gt;<br></tt><br>=
<BR>
</body></html>

--1__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F--


--0__=8FBBF5BDDFFCB59F8f9e8a93df938690918c8FBBF5BDDFFCB59F--