Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
From: Matthias Boehm <mboehm7@googlemail.com>
Subject: Re: [DISCUSS] Adding tensorboard-like functionality to SystemML
To: dev@systemml.incubator.apache.org
References: <OFCA22C68F.166042C1-ON00258059.007EC423-88258059.00803002@notes.na.collabserv.com>
Message-ID: <d099ac51-648d-caf6-4e14-7b3d0a42f9e5@gmail.com>
Date: Fri, 28 Oct 2016 10:18:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <OFCA22C68F.166042C1-ON00258059.007EC423-88258059.00803002@notes.na.collabserv.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
archived-at: Fri, 28 Oct 2016 08:24:04 -0000

Thanks for putting this together Niketan. However, could we please 
postpone this discussion after our 1.0 release? Right now, I'm concerned 
to see that we're adding many experimental features without really 
getting them done. This includes for example, the GPU backend, the new 
MLContext API, the Python DSL, the deep learning builtin functions, the 
Scala algorithm wrappers, the old Spark debugger interface, and 
compressed linear algebra. I think we should finish these features first 
before moving on. If we're not careful about that, it would quickly 
create a very bad impression for new users.

Regards,
Matthias

On 10/28/2016 1:20 AM, Niketan Pansare wrote:
>
>
> Hi all,
>
> To give every context, I am working on a new deep learning API for SystemML
> that is backed by the NN library (
> https://github.com/apache/incubator-systemml/tree/master/scripts/staging/SystemML-NN/nn
> ). This API allows the users to express their model using Caffe
> specification and perform fit/predict similar to scikit-learn APIs. I have
> created a sample notebook explaining the usage of the API:
> https://github.com/niketanpansare/incubator-systemml/blob/1b655ebeec6cdffd66b282eadc4810ecfd39e4f2/samples/jupyter-notebooks/Barista-API-Demo.ipynb
> . This API also allows the user to load and store pre-trained models. See
> https://github.com/niketanpansare/model_zoo/tree/master/caffe/vision/vgg/ilsvrc12
>
> As part of this API, I added a mini-tensorboard like functionality (see
> step 6 and 7) using matplotlib. If there is enough interest, we can extend
> and standardize the visualization functionality across all over algorithms.
> Here are some initial discussion points:
> 1. Primary visualization mechanism (Jupyter or a standalone app or both =>
> former is useful for cloud offering such as DSX and latter provides the
> design team more creative control)
> 2. What to plot for each algorithm (data scientists and algorithms
> developers will help us here).
> 3. Standardize UI (if we decide to go with Jupyter, we need to extend the
> code in _visualize method:
> https://github.com/niketanpansare/incubator-systemml/blob/1b655ebeec6cdffd66b282eadc4810ecfd39e4f2/src/main/python/systemml/mllearn/estimators.py#L621
> )
> 4. Primary APIs to target (python, scala, command-line or all)
>
> Thanks,
>
> Niketan Pansare
> IBM Almaden Research Center
> E-mail: npansar At us.ibm.com
> http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
>