Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
To: dev@systemml.incubator.apache.org
In-Reply-To: <d099ac51-648d-caf6-4e14-7b3d0a42f9e5@gmail.com>
Date: Fri, 28 Oct 2016 15:13:18 +0000
From: "Niketan Pansare" <npansar@us.ibm.com>
Subject: Re: [DISCUSS] Adding tensorboard-like functionality to SystemML
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary=Apple-Mail-B508B6F5-8BC0-465C-9968-5E15F31F5131
Message-Id: <OFEF77D3D7.3185B894-ON0025805A.00539D67-1477667598002@notes.na.collabserv.com>
archived-at: Fri, 28 Oct 2016 15:13:33 -0000

--Apple-Mail-B508B6F5-8BC0-465C-9968-5E15F31F5131
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi Matthias,

Thanks for your feedback.

There is a tradeoff between keeping a feature in-house until it is stable, =
v/s continually getting community feedback as the work is getting done via =
PR and discussions. I am for the latter as it encourages community feedback=
 as well as participation.

I agree that our goal should be to complete the features you mentioned asap=
 and yes, we are working hard towards making the GPU backend, the deep lear=
ning built-in functions and the algorithm wrappers (ones that are already a=
dded) to be 'non-experimental' in the 1.0 release :) ... Also, like you hin=
ted, it is important to explicitly mark the experimental features in the do=
cumentation to avoid the 'bad impression'. The Python DSL will remain exper=
imental until there is more interest from the community. I am fine with del=
eting the debugger since it is rarely used, if at all.

Keeping inline with the Apache guidelines, this discussion is to allow comm=
unity to decide on whether SystemML community should consider adding new vi=
sualization functionality (since this feature is user facing). If there is =
no interest, we can either postpone or discard this discussion :)

Thanks,

Niketan.

> On Oct 28, 2016, at 1:24 AM, Matthias Boehm <mboehm7@googlemail.com> wrot=
e:
>=20
> Thanks for putting this together Niketan. However, could we please=20
> postpone this discussion after our 1.0 release? Right now, I'm concerned =

> to see that we're adding many experimental features without really=20
> getting them done. This includes for example, the GPU backend, the new=20
> MLContext API, the Python DSL, the deep learning builtin functions, the=20
> Scala algorithm wrappers, the old Spark debugger interface, and=20
> compressed linear algebra. I think we should finish these features first =

> before moving on. If we're not careful about that, it would quickly=20
> create a very bad impression for new users.
>=20
> Regards,
> Matthias
>=20
>> On 10/28/2016 1:20 AM, Niketan Pansare wrote:
>>=20
>>=20
>> Hi all,
>>=20
>> To give every context, I am working on a new deep learning API for Syste=
mML
>> that is backed by the NN library (
>> https://github.com/apache/incubator-systemml/tree/master/scripts/staging=
/SystemML-NN/nn
>> ). This API allows the users to express their model using Caffe
>> specification and perform fit/predict similar to scikit-learn APIs. I ha=
ve
>> created a sample notebook explaining the usage of the API:
>> https://github.com/niketanpansare/incubator-systemml/blob/1b655ebeec6cdf=
fd66b282eadc4810ecfd39e4f2/samples/jupyter-notebooks/Barista-API-Demo.ipynb
>> . This API also allows the user to load and store pre-trained models. See
>> https://github.com/niketanpansare/model=5Fzoo/tree/master/caffe/vision/v=
gg/ilsvrc12
>>=20
>> As part of this API, I added a mini-tensorboard like functionality (see
>> step 6 and 7) using matplotlib. If there is enough interest, we can exte=
nd
>> and standardize the visualization functionality across all over algorith=
ms.
>> Here are some initial discussion points:
>> 1. Primary visualization mechanism (Jupyter or a standalone app or both =
=3D>
>> former is useful for cloud offering such as DSX and latter provides the
>> design team more creative control)
>> 2. What to plot for each algorithm (data scientists and algorithms
>> developers will help us here).
>> 3. Standardize UI (if we decide to go with Jupyter, we need to extend the
>> code in =5Fvisualize method:
>> https://github.com/niketanpansare/incubator-systemml/blob/1b655ebeec6cdf=
fd66b282eadc4810ecfd39e4f2/src/main/python/systemml/mllearn/estimators.py#L=
621
>> )
>> 4. Primary APIs to target (python, scala, command-line or all)
>>=20
>> Thanks,
>>=20
>> Niketan Pansare
>> IBM Almaden Research Center
>> E-mail: npansar At us.ibm.com
>> http://researcher.watson.ibm.com/researcher/view.php?person=3Dus-npansar
>>=20
>=20


--Apple-Mail-B508B6F5-8BC0-465C-9968-5E15F31F5131--