systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guobao Li <guobao1993...@gmail.com>
Subject Re: Questions about MNIST LeNet example
Date Thu, 10 May 2018 21:44:14 GMT
Thanks Matthias! It will be great for passing the model to the paramserv
function.

Regards,
Guobao

2018-05-10 21:47 GMT+02:00 Matthias Boehm <mboehm7@gmail.com>:

> just FYI: we now have support for list and named-list data types in
> SystemML, which allow passing the entire model as a single handle. For
> example, you can define the following
>
> l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or
> l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4)
>
> and access entries via l1[7] or l2['g'] accordingly. We're still
> working on additional features to make the integration with IPA,
> functions, and size/type propagation smoother, but the basic
> functionality is already available.
>
> Regards,
> Matthias
>
> On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <mboehm7@gmail.com> wrote:
> > Hi Guobao,
> >
> > that sounds very good. In general, the "model" refers to the
> > collection of all weights and bias matrices of a given architecture.
> > Similar to a classic regression model, we can view the weights as the
> > "slope", i.e., multiplicative terms, while the biases are the
> > "intercept", i.e., additive terms that shift the layer output. Both
> > are subject to training and thus part of the model.
> >
> > This implies that the number of matrices in the model depends on the
> > architecture. Hence, we have two choices here: (a) allow for a
> > variable number of inputs and outputs, or (b) create a struct-like
> > data type that allows passing the collection of matrices via a single
> > handle. We've discussed the second option in other contexts as well
> > because this would also be useful for reducing the number of
> > parameters passed through function calls. I'm happy to help out
> > integrating these struct-like data types if needed.
> >
> > Great to see that you're in the process of updating the related JIRAs.
> > Let us know whenever you think you're ready with an initial draft -
> > then I'd make a detailed pass over it.
> >
> > Furthermore, I would recommend to experiment with running these
> > existing mnist lenet examples (which is one of our baselines moving
> > forward):
> > * Download the "infinite MNIST" data generator
> > (http://leon.bottou.org/projects/infimnist), and generate a moderately
> > sized dataset (e.g., 256K instances).
> > * Convert the input into SystemML's binary block format. The generator
> > produces the data in libsvm format and we provide a data converter
> > (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our
> > internal binary representation.
> > * Run the basic mnist lenet example for a few epochs.
> > * Install the native BLAS libraries mkl or openblas and try using it
> > for the above example to ensure its setup and configured correctly.
> >
> >
> > Regards,
> > Matthias
> >
> > On Sun, May 6, 2018 at 3:24 AM, Guobao Li <guobao1993.li@gmail.com>
> wrote:
> >> Hi Matthias,
> >>
> >> I'm currently reading the dml script MNIST LeNet example and got some
> >> questions. I hope that you could help me out of them.
> >>
> >> 1) Is it possible to define a matrix containing the variables? Because
> I'm
> >> wondering how to represent the model as a parameter for the "paramserv"
> >> function.
> >> 2) What is the role of bias? Why we need it?
> >>
> >> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and
> hope
> >> to get some feedback. Thanks!
> >>
> >> Regards,
> >> Guobao
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message