Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
In-Reply-To: <OF34EDC8FE.F909F10A-ON00257FEA.0029B510-88257FEA.002A795E@notes.na.collabserv.com>
References: <OF16E48056.94424DF4-ON00257FE9.005B9903-80257FE9.005BE51E@notes.na.collabserv.com> <OF34EDC8FE.F909F10A-ON00257FEA.0029B510-88257FEA.002A795E@notes.na.collabserv.com>
To: dev@systemml.incubator.apache.org
MIME-Version: 1.0
Subject: Re: Restricted Boltzmann Machine scripts
From: Nikolay Manchev <NManchev@uk.ibm.com>
Date: Fri, 8 Jul 2016 10:36:51 +0100
Content-Type: multipart/alternative; boundary="=_alternative 0034CF6580257FEA_="
Message-Id: <OFA0E585BE.B6927364-ON00257FEA.0034BF9F-80257FEA.0034CFC6@notes.na.collabserv.com>
archived-at: Fri, 08 Jul 2016 09:37:14 -0000

--=_alternative 0034CF6580257FEA_=
Content-Type: text/plain; charset="US-ASCII"

Hi Matthias,
 
Thank you for your feedback. I've just created a PR as requested.
 
Regards
Nikolay


"Matthias Boehm" <mboehm@us.ibm.com> wrote on 08/07/2016 08:43:55:

> From: "Matthias Boehm" <mboehm@us.ibm.com>
> To: dev@systemml.incubator.apache.org
> Date: 08/07/2016 08:44
> Subject: Re: Restricted Boltzmann Machine scripts
> 
> thanks for reaching out Nikolay,
> 
> 1) Scripts: Could you please create a PR to add them to /scripts/
> staging? This is the place we typically use to share new scripts. 
> Once they are tested for accuracy and runtime, we would migrate them
> into scripts/algorithms along with some basic documentation. Thanks.
> 
> 2) Matrix/Vector element-wise binary operations: As Fred already 
> mentioned, SystemML supports both Matrix-Row Vector and Matrix-
> Column Vector operations, where the right-hand side input is 
> logically replicated which often allows us to broadcast the vector 
> and hence avoid shuffle operations. However, if you prefer to write 
> it explicitly as repmat like X %*% (v %*% matrix(1, 1, ncols(X))), 
> that's fine too - the SystemML compiler would then anyway 
> automatically rewrite this to matrix-vector operations if the repmat
> is in the same basic block of statements.
> 
> Regards,
> Matthias
> 
> 
> [image removed] Nikolay Manchev ---07/07/2016 09:44:28 AM---Hi all, 
> I am dropping a note as suggested by Fred below.
> 
> From: Nikolay Manchev <NManchev@uk.ibm.com>
> To: dev@systemml.incubator.apache.org
> Date: 07/07/2016 09:44 AM
> Subject: Restricted Boltzmann Machine scripts
> 
> 
> 
> Hi all,
> 
> I am dropping a note as suggested by Fred below.
> 
> I wrote a couple of DML scripts that implement Restricted Boltzmann 
> Machines, which are available on GitHub. This one uses CD-1 mini-batch 
> training to fit the model (weights and biases): 
> https://github.com/nmanchev/incubator-systemml/blob/neuralnets/
> scripts/algorithms/rbm_minibatch.dml
> 
> This one runs a data sets through the trained RBM and outputs sample of 
> P(h=1|v) for each observation: 
> https://github.com/nmanchev/incubator-systemml/blob/neuralnets/
> scripts/algorithms/rbm_run.dml
> 
> Can you please add them to scripts/algorithms? I also created a JIRA 
issue 
> for adding them here: https://issues.apache.org/jira/browse/SYSTEMML-777
> 
> Many thanks
> Nikolay
> 
> 
> ----- Original message -----
> From: Frederick R Reiss/Almaden/IBM
> To: Nikolay Manchev/UK/IBM@IBMGB
> Cc: dev@systemml.incubator.apache.org
> Subject: Re: Question on SystemML - RBMs and repmat()
> Date: Thu, Jul 7, 2016 3:07 PM
> 
> Hi Nikolay,
> 
> I appreciate your interest in the project. To answer your question: You 
> should be able to write "X%*%W + B" and get the semantics you want. The 
> SystemML compiler automatically pads vectors with copies of themselves 
> when it sees a cellwise operation between a matrix and a vector. So if 
you 
> run the DML code:
> 
> 
>    A = matrix (1.0, rows=3, cols=3)
>    v = matrix (2.0, rows=1, cols=3)
>    sum = A + v
>    print(toString(sum))
> 
> the output will be:
> 
>    3.000 3.000 3.000
>    3.000 3.000 3.000
>    3.000 3.000 3.000
> 
> 
> Exposing cellwise matrix-vector operations to the SystemML optimizer in 
> this way should result in more efficient parallel plans, since it's 
easier 
> for the optimizer to detect that it can broadcast the vector and stream 
> the matrix.
> 
> The PNMF script on the SystemML home page (http://systemml.apache.org) 
has 
> a more in-depth example of the same pattern:
> 
>    while (iter < max_iterations) {
>     iter = iter + 1;
>     H = (H * (t(W) %*% (V/(W%*%H)))) / t(colSums(W));
>     W = (W * ((V/(W%*%H)) %*% t(H))) / t(rowSums(H));
>     obj = as.scalar(colSums(W) %*% rowSums(H)) - sum(V * log(W%*%H));
>     print("iter=" + iter + " obj=" + obj);
>    }
> 
> The part in red divides the matrix (H * (t(W) %*% (V/(W%*%H)))) by the 
> vector t(colSums(W)). In R, the divisor in this expression would need to 

> be (t(matrix(colSums(W),nrow=1))%*%matrix(rep(1,m),nrow=1)) or something 

> equivalent.
> 
> 
> 
> I think that an example script for training Boltzmann machines would be 
a 
> useful addition to the SystemML distribution. Would you mind opening a 
> JIRA issue for adding this script and posting a link to the JIRA on the 
> SystemML mailing list? Our JIRA instance is at 
> https://issues.apache.org/jira/browse/SYSTEMML, and our mailing list is 
at 
> http://systemml.apache.org/community. By the way, it's good to post 
> questions like your question below the mailing list so that others who 
run 
> into the same issue will have an easier time finding the solution; I'm 
> CCing the list with my response here.
> 
> Fred
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 

> 741598. 
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
> 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

--=_alternative 0034CF6580257FEA_=--