systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm" <>
Subject Re: POC Eclipse IDE for DML
Date Sun, 03 Jan 2016 00:33:42 GMT

well, in general all of our builtin functions are handled in
BuiltinFunctionExpression - so that's the place to look at if the language
reference is not enough. The only reason why as.scalar and similar
functions appear in the grammar is a workaround for functions which violate
our own function naming convention (i.e., user-defined functions must not
contain a '.'). However, this was a compromise in favor of users familiar
with R (which provides functions like as.integer, as.logical, as.matrix etc
as well).


From:	Deron Eriksson <>
Date:	01/02/2016 04:24 PM
Subject:	Re: POC Eclipse IDE for DML

Hi Matthias,

Thank you for the clarification with regards to the parser grammar and
built-in functions. To me, as a relative newbie, in one way, the built-in
functions seem similar to java.lang.* classes in Java, but in another way,
they seem almost even more intrinsic than that.

I guess what I was mainly trying to get at is that Dml.g4 is fantastic in
terms of flexibility, especially when considered in conjunction with the
supporting referenced Java classes, but Nakul's grammar for IDE support is
able to convey the essence of the language capabilities (such as built-in
functions) in one place.

Also, consider the following. The icdf() function is not in Dml.g4 since it
is a built-in function. It is handled in the
parser.ParameterizedBuiltinFunctionExpression class. On the other hand,
as.scalar() is in Dml.g4 and in parser.BuiltinFunctionExpression. However,
castAsScalar() is not in Dml.g4 but is handled similarly in

As a result of this, I am now confused. If I am trying to understand what I
can do with the language, do I look in Dml.g4 (such as for as.scalar()), do
I look in BuiltinFunctionExpression (such as for as.scalar() and
castAsScalar()), or do I look in ParameterizedBuiltinFunctionExpression
(such as for icdf())?

Perhaps as.scalar() needs to be removed from the grammar and as.scalar()
and castAsScalar() need to be condensed to one function and I will be less
confused. :-)

Maybe I am overthinking things. The DML Language Reference does contain a
tremendous amount of useful information with regards to the language


On Sat, Jan 2, 2016 at 12:25 PM, Matthias Boehm <> wrote:

> just to clarify, our parser grammar does not include builtin functions
> because they are -- in contrast to keywords and language constructs --
> part of the DML/PyDML syntax. This is important for both maintainability
> and flexibility. For example, it allows you to define a variable or
> user-defined function with the name 't' although there exists a builtin
> function t() for transpose operations.
> Regards,
> Matthias
> [image: Inactive hide details for Deron Eriksson ---01/01/2016 11:38:27
> AM---Hi Nakul, This is very cool! I see great value in IDE tool]Deron
> Eriksson ---01/01/2016 11:38:27 AM---Hi Nakul, This is very cool! I see
> great value in IDE tools such as an Eclipse editor
> From: Deron Eriksson <>
> To:
> Date: 01/01/2016 11:38 AM
> Subject: Re: POC Eclipse IDE for DML
> ------------------------------
> Hi Nakul,
> This is very cool! I see great value in IDE tools such as an Eclipse
> for DML/PyDML. Highlighting syntax errors helps people catch syntax
> mistakes before running the script, and IDE code completion is a great
> productivity boost.
> Also, I like how clean and readable the grammar is. I don't know how
> feel, but it is so clean that, if flushed out, it might be a useful
> addition to the DML Language Reference. It does things such as enumerate
> the built-in functions, which is handy when trying to wrap your mind
> the language. The Dml.g4 file is very powerful and useful but I have a
> harder time understanding the language syntax when looking at it since
> things like built-in functions aren't included in it. I wonder if there
> a way to integrate some of your work into Dml.g4?
> I haven't done Xtext before. Do you have any advice to get started if I'd
> like to try this out in my Eclipse (for instance, to look at the
> issue)?
> Deron
> On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <> wrote:
> > Hi,
> >
> > I've been trying to build a Proof of Concept IDE for DML using Xtext.
> >
> >
> >
> > Grammar File :
> >
> >

> >
> > Here are some screenshots:
> >
> >
> > The files shown are from the algorithms folder.
> >
> >
> > (There was no particular reason to have chosen these files over others)
> >
> > For this POC, my only goal was to get syntax highlighting to work in
> > Eclipse.
> > SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3.
> Some
> > amount of work was needed on the grammar before it could be used it for
> > Xtext.
> >
> > From the screenshots, you can see that eclipse thinks there are
> > errors in the files. This has to do with statements ending in an
> > semicolon.
> >
> > I only supplied xtext a grammar file from which to generate the IDE. I
> have
> > not yet worked on the generated code to add any other features.
> > I could not come up with a LL(*) grammar that would let me express
> optional
> > semicolons at the end of statements. For the time being, the grammar
> > requires semicolons at the end of statements (as you can see in the
> grammar
> > file :
> >
> >

> > )
> >
> > I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> > dealing with this (assuming no way of modifying the grammar to fix
> >
> > 1. Support a subset of DML in the XText based tools
> > 2. Modify the xtext generated tools to fix this
> > 3. Write an eclipse plugin from scratch with complete control over
> > and parser (among other components)
> >
> > Option 1 requires the least amount of work.
> > I am not sure between options 2 & 3, which would be more work.
> >
> >
> > Is this useful to anyone?
> > If so, are there any thoughts or suggestions on how to approach the
> > optional semicolon problem?
> >
> >
> > Nakul Jindal
> >
> >

  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message