systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolay Manchev <NManc...@uk.ibm.com>
Subject Restricted Boltzmann Machine scripts
Date Thu, 07 Jul 2016 16:43:44 GMT
Hi all,

I am dropping a note as suggested by Fred below.

I wrote a couple of DML scripts that implement Restricted Boltzmann 
Machines, which are available on GitHub. This one uses CD-1 mini-batch 
training to fit the model (weights and biases): 
https://github.com/nmanchev/incubator-systemml/blob/neuralnets/scripts/algorithms/rbm_minibatch.dml

This one runs a data sets through the trained RBM and outputs sample of 
P(h=1|v) for each observation: 
https://github.com/nmanchev/incubator-systemml/blob/neuralnets/scripts/algorithms/rbm_run.dml

Can you please add them to scripts/algorithms? I also created a JIRA issue 
for adding them here: https://issues.apache.org/jira/browse/SYSTEMML-777

Many thanks
Nikolay


----- Original message -----
From: Frederick R Reiss/Almaden/IBM
To: Nikolay Manchev/UK/IBM@IBMGB
Cc: dev@systemml.incubator.apache.org
Subject: Re: Question on SystemML - RBMs and repmat()
Date: Thu, Jul 7, 2016 3:07 PM
 
Hi Nikolay,

I appreciate your interest in the project. To answer your question: You 
should be able to write "X%*%W + B" and get the semantics you want. The 
SystemML compiler automatically pads vectors with copies of themselves 
when it sees a cellwise operation between a matrix and a vector. So if you 
run the DML code:


    A = matrix (1.0, rows=3, cols=3)
    v = matrix (2.0, rows=1, cols=3)
    sum = A + v
    print(toString(sum))

the output will be:

    3.000 3.000 3.000
    3.000 3.000 3.000
    3.000 3.000 3.000


Exposing cellwise matrix-vector operations to the SystemML optimizer in 
this way should result in more efficient parallel plans, since it's easier 
for the optimizer to detect that it can broadcast the vector and stream 
the matrix.

The PNMF script on the SystemML home page (http://systemml.apache.org) has 
a more in-depth example of the same pattern:

    while (iter < max_iterations) {
     iter = iter + 1;
     H = (H * (t(W) %*% (V/(W%*%H)))) / t(colSums(W));
     W = (W * ((V/(W%*%H)) %*% t(H))) / t(rowSums(H));
     obj = as.scalar(colSums(W) %*% rowSums(H)) - sum(V * log(W%*%H));
     print("iter=" + iter + " obj=" + obj);
    }

The part in red divides the matrix (H * (t(W) %*% (V/(W%*%H)))) by the 
vector t(colSums(W)). In R, the divisor in this expression would need to 
be (t(matrix(colSums(W),nrow=1))%*%matrix(rep(1,m),nrow=1)) or something 
equivalent.

 

I think that an example script for training Boltzmann machines would be a 
useful addition to the SystemML distribution. Would you mind opening a 
JIRA issue for adding this script and posting a link to the JIRA on the 
SystemML mailing list? Our JIRA instance is at 
https://issues.apache.org/jira/browse/SYSTEMML, and our mailing list is at 
http://systemml.apache.org/community. By the way, it's good to post 
questions like your question below the mailing list so that others who run 
into the same issue will have an easier time finding the solution; I'm 
CCing the list with my response here.

Fred
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message