systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm" <mbo...@us.ibm.com>
Subject Re: Restricted Boltzmann Machine scripts
Date Fri, 08 Jul 2016 07:43:55 GMT

thanks for reaching out Nikolay,

1) Scripts: Could you please create a PR to add them to /scripts/staging?
This is the place we typically use to share new scripts. Once they are
tested for accuracy and runtime, we would migrate them into
scripts/algorithms along with some basic documentation. Thanks.

2) Matrix/Vector element-wise binary operations: As Fred already mentioned,
SystemML supports both Matrix-Row Vector and Matrix-Column Vector
operations, where the right-hand side input is logically replicated which
often allows us to broadcast the vector and hence avoid shuffle operations.
However, if you prefer to write it explicitly as repmat like X %*% (v %*%
matrix(1, 1, ncols(X))), that's fine too - the SystemML compiler would then
anyway automatically rewrite this to matrix-vector operations if the repmat
is in the same basic block of statements.

Regards,
Matthias




From:	Nikolay Manchev <NManchev@uk.ibm.com>
To:	dev@systemml.incubator.apache.org
Date:	07/07/2016 09:44 AM
Subject:	Restricted Boltzmann Machine scripts



Hi all,

I am dropping a note as suggested by Fred below.

I wrote a couple of DML scripts that implement Restricted Boltzmann
Machines, which are available on GitHub. This one uses CD-1 mini-batch
training to fit the model (weights and biases):
https://github.com/nmanchev/incubator-systemml/blob/neuralnets/scripts/algorithms/rbm_minibatch.dml


This one runs a data sets through the trained RBM and outputs sample of
P(h=1|v) for each observation:
https://github.com/nmanchev/incubator-systemml/blob/neuralnets/scripts/algorithms/rbm_run.dml


Can you please add them to scripts/algorithms? I also created a JIRA issue
for adding them here: https://issues.apache.org/jira/browse/SYSTEMML-777

Many thanks
Nikolay


----- Original message -----
From: Frederick R Reiss/Almaden/IBM
To: Nikolay Manchev/UK/IBM@IBMGB
Cc: dev@systemml.incubator.apache.org
Subject: Re: Question on SystemML - RBMs and repmat()
Date: Thu, Jul 7, 2016 3:07 PM

Hi Nikolay,

I appreciate your interest in the project. To answer your question: You
should be able to write "X%*%W + B" and get the semantics you want. The
SystemML compiler automatically pads vectors with copies of themselves
when it sees a cellwise operation between a matrix and a vector. So if you
run the DML code:


    A = matrix (1.0, rows=3, cols=3)
    v = matrix (2.0, rows=1, cols=3)
    sum = A + v
    print(toString(sum))

the output will be:

    3.000 3.000 3.000
    3.000 3.000 3.000
    3.000 3.000 3.000


Exposing cellwise matrix-vector operations to the SystemML optimizer in
this way should result in more efficient parallel plans, since it's easier
for the optimizer to detect that it can broadcast the vector and stream
the matrix.

The PNMF script on the SystemML home page (http://systemml.apache.org) has
a more in-depth example of the same pattern:

    while (iter < max_iterations) {
     iter = iter + 1;
     H = (H * (t(W) %*% (V/(W%*%H)))) / t(colSums(W));
     W = (W * ((V/(W%*%H)) %*% t(H))) / t(rowSums(H));
     obj = as.scalar(colSums(W) %*% rowSums(H)) - sum(V * log(W%*%H));
     print("iter=" + iter + " obj=" + obj);
    }

The part in red divides the matrix (H * (t(W) %*% (V/(W%*%H)))) by the
vector t(colSums(W)). In R, the divisor in this expression would need to
be (t(matrix(colSums(W),nrow=1))%*%matrix(rep(1,m),nrow=1)) or something
equivalent.



I think that an example script for training Boltzmann machines would be a
useful addition to the SystemML distribution. Would you mind opening a
JIRA issue for adding this script and posting a link to the JIRA on the
SystemML mailing list? Our JIRA instance is at
https://issues.apache.org/jira/browse/SYSTEMML, and our mailing list is at
http://systemml.apache.org/community. By the way, it's good to post
questions like your question below the mailing list so that others who run
into the same issue will have an easier time finding the solution; I'm
CCing the list with my response here.

Fred
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message