systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Niketan Pansare" <npan...@us.ibm.com>
Subject RE: How to contribute SystemML
Date Wed, 02 Dec 2015 17:29:38 GMT

Hi Tatsuya,

Thanks for your interest in the SystemML project. Just so that we are on
same page, there are two interpretation of REPL with respect to SystemML:
1. Using SystemML in Spark REPL.
- Scala Spark-shell:
http://apache.github.io/incubator-systemml/mlcontext-programming-guide.html
. This also involves the integration with MLPipeline (
https://github.com/apache/incubator-systemml/blob/master/src/main/java/com/ibm/bi/dml/api/ml/LogisticRegression.java
) APIs.
- PySpark shell (using SystemML.py)
- Jupyter (using SystemML.py)

2. Having a "DML kernel/interpreter".
- A DML kernel for Jupyter (will require a command-line interpreter)
- A command-line interpreter where data scientists can type DML commands.

Like Matthias, I would recommend that implementing an interpreter for DML
might not be a good task to explore SystemML. Still, I will try to give you
an overview and please pardon me for hand-waving a bit in below paragraph
for sake of simplicity.

We do have a "mini-interpreter" in the form of a debugger and it might be a
good place to explore before implementing an interpreter. Please note: the
debugger works at level of instruction, not at the level of DML statement.
Also, the debugger takes as input a DML script and hence SystemML's
optimizer has full scope for optimization. However, to allow for tasks such
as "step into", we disable certain optimizations and have a special level
for the debugger (O5_DEBUG_MODE). For example: algrebraic simplification,
interprocedural analysis, branch removal, dynamic recompilations, sum
product rewrites, etc are disabled. Browsing through the debugger code
should give you a fair idea about the machinery required in SystemML for
implementing a full-fledged interpreter (such as runtime instructions,
symbol table, instructions-to-DML mapping, piggybacking, etc). Then one
must step back and look at the high-level design decisions as well, such as
lazy evaluation.

To get started with the debugger, please look at
http://apache.github.io/incubator-systemml/debugger-guide.html and also in
the com.ibm.bi.dml.debug package.

Please note the URL might change soon when we change our package names to
org.apache.sysml.

Reference:
https://github.com/apache/incubator-systemml/blob/master/src/main/java/com/ibm/bi/dml/api/MLContext.java
https://github.com/apache/incubator-systemml/blob/master/src/main/java/com/ibm/bi/dml/api/python/SystemML.py
https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages
https://github.com/apache/incubator-systemml/tree/master/src/main/java/com/ibm/bi/dml/debug

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar



From:	"Tatsuya Nishiyama" <nishiyama.tatsuya@lab.ntt.co.jp>
To:	<dev@systemml.incubator.apache.org>
Date:	12/02/2015 01:06 AM
Subject:	RE: How to contribute SystemML



Hi Matthias,

Thanks for your reply.

> Such a feature would, however, require substantial discussions,
especially
with regard to optimization scope etc and hence might not be a good
starting
task.
> How about we select some smaller tasks on our existing APIs and once you
feel comfortable get started on a larger feature like that?
OK, I understand it. I will try to find such smaller tasks, and tackle it.
If you already know any started tasks , please tell me.

Best regards,
- Tatsuya

--

From: Matthias Boehm [mailto:mboehm@us.ibm.com]
Sent: Wednesday, December 02, 2015 4:47 PM
To: dev@systemml.incubator.apache.org
Subject: Re: How to contribute SystemML

Hi Tatsuya,

thanks for your interest, we'd love to help you get started.

Although we do have various APIs, including MLContext that allows you to
invoke DML scripts from Spark's interactive shell, we don't have an actual
REPL interface yet. Niketan built an initial prototype of a related API.
@Niketan: Do you want to comment on that?

Such a feature would, however, require substantial discussions, especially
with regard to optimization scope etc and hence might not be a good
starting
task. How about we select some smaller tasks on our existing APIs and once
you feel comfortable get started on a larger feature like that?


Regards,
Matthias

"Tatsuya Nishiyama" ---12/01/2015 10:50:41 PM---Hi, I'm interested in
contributing SystemML. I've checked the documentation, and

From: "Tatsuya Nishiyama" <nishiyama.tatsuya@lab.ntt.co.jp>
To: <dev@systemml.incubator.apache.org>
Date: 12/01/2015 10:50 PM
Subject: How to contribute SystemML
________________________________________



Hi,

I'm interested in contributing SystemML. I've checked the documentation,
and
it looks like SystemML doesn't have REPL for interactive execution. Is it
correct? If SystemML doesn't have REPL, I would like to contribute it. BTW,
JIRA looks to be unavailable for now:
https://issues.apache.org/jira/browse/SYSTEMML
How can I start contribution?

Thanks,
- Tatsuya





Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message