systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tatsuya Nishiyama" <nishiyama.tats...@lab.ntt.co.jp>
Subject RE: How to contribute SystemML
Date Thu, 03 Dec 2015 05:20:37 GMT
Hi Niketan,

Thank you for telling me many things. I understand the plan of SystemML to
support REPL.

> Like Matthias, I would recommend that implementing an interpreter for DML
might not be a good task to explore SystemML.
I will start contributing SystemML from small tasks. 

> We do have a "mini-interpreter" in the form of a debugger and it might be
a good place to explore before implementing an interpreter.
This is so great. I will try to use it and read the source code.  Thank you
again for your giving me the overview and some references. 

Thanks, 
- Tatsuya

--
From: Niketan Pansare [mailto:npansar@us.ibm.com] 
Sent: Thursday, December 03, 2015 2:30 AM
To: dev@systemml.incubator.apache.org
Subject: RE: How to contribute SystemML

Hi Tatsuya,

Thanks for your interest in the SystemML project. Just so that we are on
same page, there are two interpretation of REPL with respect to SystemML:
1. Using SystemML in Spark REPL.
- Scala Spark-shell:
http://apache.github.io/incubator-systemml/mlcontext-programming-guide.html.
This also involves the integration with MLPipeline
(https://github.com/apache/incubator-systemml/blob/master/src/main/java/com/
ibm/bi/dml/api/ml/LogisticRegression.java) APIs.
- PySpark shell (using SystemML.py)
- Jupyter (using SystemML.py)

2. Having a "DML kernel/interpreter".
- A DML kernel for Jupyter (will require a command-line interpreter)
- A command-line interpreter where data scientists can type DML commands. 

Like Matthias, I would recommend that implementing an interpreter for DML
might not be a good task to explore SystemML. Still, I will try to give you
an overview and please pardon me for hand-waving a bit in below paragraph
for sake of simplicity.

We do have a "mini-interpreter" in the form of a debugger and it might be a
good place to explore before implementing an interpreter. Please note: the
debugger works at level of instruction, not at the level of DML statement.
Also, the debugger takes as input a DML script and hence SystemML's
optimizer has full scope for optimization. However, to allow for tasks such
as "step into", we disable certain optimizations and have a special level
for the debugger (O5_DEBUG_MODE). For example: algrebraic simplification,
interprocedural analysis, branch removal, dynamic recompilations, sum
product rewrites, etc are disabled. Browsing through the debugger code
should give you a fair idea about the machinery required in SystemML for
implementing a full-fledged interpreter (such as runtime instructions,
symbol table, instructions-to-DML mapping, piggybacking, etc). Then one must
step back and look at the high-level design decisions as well, such as lazy
evaluation.

To get started with the debugger, please look at
http://apache.github.io/incubator-systemml/debugger-guide.html and also in
the com.ibm.bi.dml.debug package.

Please note the URL might change soon when we change our package names to
org.apache.sysml.

Reference:
https://github.com/apache/incubator-systemml/blob/master/src/main/java/com/i
bm/bi/dml/api/MLContext.java
https://github.com/apache/incubator-systemml/blob/master/src/main/java/com/i
bm/bi/dml/api/python/SystemML.py
https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages
https://github.com/apache/incubator-systemml/tree/master/src/main/java/com/i
bm/bi/dml/debug

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar

"Tatsuya Nishiyama" ---12/02/2015 01:06:33 AM---Hi Matthias, Thanks for your
reply.

From: "Tatsuya Nishiyama" <nishiyama.tatsuya@lab.ntt.co.jp>
To: <dev@systemml.incubator.apache.org>
Date: 12/02/2015 01:06 AM
Subject: RE: How to contribute SystemML
________________________________________



Hi Matthias,

Thanks for your reply.

> Such a feature would, however, require substantial discussions, especially
with regard to optimization scope etc and hence might not be a good starting
task.
> How about we select some smaller tasks on our existing APIs and once you
feel comfortable get started on a larger feature like that?
OK, I understand it. I will try to find such smaller tasks, and tackle it.
If you already know any started tasks , please tell me.

Best regards,
- Tatsuya

-- 

From: Matthias Boehm [mailto:mboehm@us.ibm.com] 
Sent: Wednesday, December 02, 2015 4:47 PM
To: dev@systemml.incubator.apache.org
Subject: Re: How to contribute SystemML

Hi Tatsuya,

thanks for your interest, we'd love to help you get started. 

Although we do have various APIs, including MLContext that allows you to
invoke DML scripts from Spark's interactive shell, we don't have an actual
REPL interface yet. Niketan built an initial prototype of a related API.
@Niketan: Do you want to comment on that? 

Such a feature would, however, require substantial discussions, especially
with regard to optimization scope etc and hence might not be a good starting
task. How about we select some smaller tasks on our existing APIs and once
you feel comfortable get started on a larger feature like that? 


Regards,
Matthias

"Tatsuya Nishiyama" ---12/01/2015 10:50:41 PM---Hi, I'm interested in
contributing SystemML. I've checked the documentation, and

From: "Tatsuya Nishiyama" <nishiyama.tatsuya@lab.ntt.co.jp>
To: <dev@systemml.incubator.apache.org>
Date: 12/01/2015 10:50 PM
Subject: How to contribute SystemML
________________________________________



Hi,

I'm interested in contributing SystemML. I've checked the documentation, and
it looks like SystemML doesn't have REPL for interactive execution. Is it
correct? If SystemML doesn't have REPL, I would like to contribute it. BTW,
JIRA looks to be unavailable for now:
https://issues.apache.org/jira/browse/SYSTEMML
How can I start contribution?

Thanks,
- Tatsuya






Mime
View raw message