systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <mboe...@gmail.com>
Subject Re: Questions about MNIST LeNet example
Date Thu, 10 May 2018 19:47:24 GMT
just FYI: we now have support for list and named-list data types in
SystemML, which allow passing the entire model as a single handle. For
example, you can define the following

l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or
l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4)

and access entries via l1[7] or l2['g'] accordingly. We're still
working on additional features to make the integration with IPA,
functions, and size/type propagation smoother, but the basic
functionality is already available.

Regards,
Matthias

On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <mboehm7@gmail.com> wrote:
> Hi Guobao,
>
> that sounds very good. In general, the "model" refers to the
> collection of all weights and bias matrices of a given architecture.
> Similar to a classic regression model, we can view the weights as the
> "slope", i.e., multiplicative terms, while the biases are the
> "intercept", i.e., additive terms that shift the layer output. Both
> are subject to training and thus part of the model.
>
> This implies that the number of matrices in the model depends on the
> architecture. Hence, we have two choices here: (a) allow for a
> variable number of inputs and outputs, or (b) create a struct-like
> data type that allows passing the collection of matrices via a single
> handle. We've discussed the second option in other contexts as well
> because this would also be useful for reducing the number of
> parameters passed through function calls. I'm happy to help out
> integrating these struct-like data types if needed.
>
> Great to see that you're in the process of updating the related JIRAs.
> Let us know whenever you think you're ready with an initial draft -
> then I'd make a detailed pass over it.
>
> Furthermore, I would recommend to experiment with running these
> existing mnist lenet examples (which is one of our baselines moving
> forward):
> * Download the "infinite MNIST" data generator
> (http://leon.bottou.org/projects/infimnist), and generate a moderately
> sized dataset (e.g., 256K instances).
> * Convert the input into SystemML's binary block format. The generator
> produces the data in libsvm format and we provide a data converter
> (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our
> internal binary representation.
> * Run the basic mnist lenet example for a few epochs.
> * Install the native BLAS libraries mkl or openblas and try using it
> for the above example to ensure its setup and configured correctly.
>
>
> Regards,
> Matthias
>
> On Sun, May 6, 2018 at 3:24 AM, Guobao Li <guobao1993.li@gmail.com> wrote:
>> Hi Matthias,
>>
>> I'm currently reading the dml script MNIST LeNet example and got some
>> questions. I hope that you could help me out of them.
>>
>> 1) Is it possible to define a matrix containing the variables? Because I'm
>> wondering how to represent the model as a parameter for the "paramserv"
>> function.
>> 2) What is the role of bias? Why we need it?
>>
>> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope
>> to get some feedback. Thanks!
>>
>> Regards,
>> Guobao
>>

Mime
View raw message