Hi all,
I am dropping a note as suggested by Fred below.
I wrote a couple of DML scripts that implement Restricted Boltzmann
Machines, which are available on GitHub. This one uses CD1 minibatch
training to fit the model (weights and biases):
https://github.com/nmanchev/incubatorsystemml/blob/neuralnets/scripts/algorithms/rbm_minibatch.dml
This one runs a data sets through the trained RBM and outputs sample of
P(h=1v) for each observation:
https://github.com/nmanchev/incubatorsystemml/blob/neuralnets/scripts/algorithms/rbm_run.dml
Can you please add them to scripts/algorithms? I also created a JIRA issue
for adding them here: https://issues.apache.org/jira/browse/SYSTEMML777
Many thanks
Nikolay
 Original message 
From: Frederick R Reiss/Almaden/IBM
To: Nikolay Manchev/UK/IBM@IBMGB
Cc: dev@systemml.incubator.apache.org
Subject: Re: Question on SystemML  RBMs and repmat()
Date: Thu, Jul 7, 2016 3:07 PM
Hi Nikolay,
I appreciate your interest in the project. To answer your question: You
should be able to write "X%*%W + B" and get the semantics you want. The
SystemML compiler automatically pads vectors with copies of themselves
when it sees a cellwise operation between a matrix and a vector. So if you
run the DML code:
A = matrix (1.0, rows=3, cols=3)
v = matrix (2.0, rows=1, cols=3)
sum = A + v
print(toString(sum))
the output will be:
3.000 3.000 3.000
3.000 3.000 3.000
3.000 3.000 3.000
Exposing cellwise matrixvector operations to the SystemML optimizer in
this way should result in more efficient parallel plans, since it's easier
for the optimizer to detect that it can broadcast the vector and stream
the matrix.
The PNMF script on the SystemML home page (http://systemml.apache.org) has
a more indepth example of the same pattern:
while (iter < max_iterations) {
iter = iter + 1;
H = (H * (t(W) %*% (V/(W%*%H)))) / t(colSums(W));
W = (W * ((V/(W%*%H)) %*% t(H))) / t(rowSums(H));
obj = as.scalar(colSums(W) %*% rowSums(H))  sum(V * log(W%*%H));
print("iter=" + iter + " obj=" + obj);
}
The part in red divides the matrix (H * (t(W) %*% (V/(W%*%H)))) by the
vector t(colSums(W)). In R, the divisor in this expression would need to
be (t(matrix(colSums(W),nrow=1))%*%matrix(rep(1,m),nrow=1)) or something
equivalent.
I think that an example script for training Boltzmann machines would be a
useful addition to the SystemML distribution. Would you mind opening a
JIRA issue for adding this script and posting a link to the JIRA on the
SystemML mailing list? Our JIRA instance is at
https://issues.apache.org/jira/browse/SYSTEMML, and our mailing list is at
http://systemml.apache.org/community. By the way, it's good to post
questions like your question below the mailing list so that others who run
into the same issue will have an easier time finding the solution; I'm
CCing the list with my response here.
Fred
Unless stated otherwise above:
IBM United Kingdom Limited  Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
