systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dusenberr...@gmail.com
Subject Re: [DISCUSS] Adding tensorboard-like functionality to SystemML
Date Sat, 29 Oct 2016 00:46:23 GMT
Visualization is a good topic to bring up for the project. I would like to add another possible
option of using TensorBoard directly. I have not looked into the file format used for TensorBoard,
but it may be possible to simple adopt that format, and simply write our stats to that type
of file. That would allow us to reuse that project without having to write our own. 

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 28, 2016, at 8:13 AM, Niketan Pansare <npansar@us.ibm.com> wrote:
> 
> Hi Matthias,
> 
> Thanks for your feedback.
> 
> There is a tradeoff between keeping a feature in-house until it is stable, v/s continually
getting community feedback as the work is getting done via PR and discussions. I am for the
latter as it encourages community feedback as well as participation.
> 
> I agree that our goal should be to complete the features you mentioned asap and yes,
we are working hard towards making the GPU backend, the deep learning built-in functions and
the algorithm wrappers (ones that are already added) to be 'non-experimental' in the 1.0 release
:) ... Also, like you hinted, it is important to explicitly mark the experimental features
in the documentation to avoid the 'bad impression'. The Python DSL will remain experimental
until there is more interest from the community. I am fine with deleting the debugger since
it is rarely used, if at all.
> 
> Keeping inline with the Apache guidelines, this discussion is to allow community to decide
on whether SystemML community should consider adding new visualization functionality (since
this feature is user facing). If there is no interest, we can either postpone or discard this
discussion :)
> 
> Thanks,
> 
> Niketan.
> 
>> On Oct 28, 2016, at 1:24 AM, Matthias Boehm <mboehm7@googlemail.com> wrote:
>> 
>> Thanks for putting this together Niketan. However, could we please 
>> postpone this discussion after our 1.0 release? Right now, I'm concerned 
>> to see that we're adding many experimental features without really 
>> getting them done. This includes for example, the GPU backend, the new 
>> MLContext API, the Python DSL, the deep learning builtin functions, the 
>> Scala algorithm wrappers, the old Spark debugger interface, and 
>> compressed linear algebra. I think we should finish these features first 
>> before moving on. If we're not careful about that, it would quickly 
>> create a very bad impression for new users.
>> 
>> Regards,
>> Matthias
>> 
>>> On 10/28/2016 1:20 AM, Niketan Pansare wrote:
>>> 
>>> 
>>> Hi all,
>>> 
>>> To give every context, I am working on a new deep learning API for SystemML
>>> that is backed by the NN library (
>>> https://github.com/apache/incubator-systemml/tree/master/scripts/staging/SystemML-NN/nn
>>> ). This API allows the users to express their model using Caffe
>>> specification and perform fit/predict similar to scikit-learn APIs. I have
>>> created a sample notebook explaining the usage of the API:
>>> https://github.com/niketanpansare/incubator-systemml/blob/1b655ebeec6cdffd66b282eadc4810ecfd39e4f2/samples/jupyter-notebooks/Barista-API-Demo.ipynb
>>> . This API also allows the user to load and store pre-trained models. See
>>> https://github.com/niketanpansare/model_zoo/tree/master/caffe/vision/vgg/ilsvrc12
>>> 
>>> As part of this API, I added a mini-tensorboard like functionality (see
>>> step 6 and 7) using matplotlib. If there is enough interest, we can extend
>>> and standardize the visualization functionality across all over algorithms.
>>> Here are some initial discussion points:
>>> 1. Primary visualization mechanism (Jupyter or a standalone app or both =>
>>> former is useful for cloud offering such as DSX and latter provides the
>>> design team more creative control)
>>> 2. What to plot for each algorithm (data scientists and algorithms
>>> developers will help us here).
>>> 3. Standardize UI (if we decide to go with Jupyter, we need to extend the
>>> code in _visualize method:
>>> https://github.com/niketanpansare/incubator-systemml/blob/1b655ebeec6cdffd66b282eadc4810ecfd39e4f2/src/main/python/systemml/mllearn/estimators.py#L621
>>> )
>>> 4. Primary APIs to target (python, scala, command-line or all)
>>> 
>>> Thanks,
>>> 
>>> Niketan Pansare
>>> IBM Almaden Research Center
>>> E-mail: npansar At us.ibm.com
>>> http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
>>> 
>> 
> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message