systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deron Eriksson <deroneriks...@gmail.com>
Subject Re: DML example on main SystemML website
Date Thu, 17 Dec 2015 21:55:05 GMT
I updated the DML example on the main site to be the PNMF example from Fred
and Matthias.

I think the idea of a DML cookbook is a fantastic idea. I can begin a
cookbook based on the examples from Shirish. Additionally, I believe Alok
Singh has several functions as a result of his algorithm development work
that could be useful in a cookbook. I like Mike's suggestion of keeping
things simple. Such a notebook would probably not have a lot of structure
at first but would gain structure over time as code snippets can be grouped
and sorted by common functionality as their numbers increase.

Deron


On Thu, Dec 17, 2015 at 12:55 AM, Berthold Reinwald <reinwald@us.ibm.com>
wrote:

> Creating a DML cookbook sounds very useful ... especially for new data
> scientists starting to pick up DML. It shows vectorization avoiding loops,
> and people are familiar with the semantics. There may be more nuggets like
> that through-out the DML algorithms. And the list will grow in the course
> of time.
>
> Regards,
> Berthold Reinwald
> IBM Almaden Research Center
> office: (408) 927 2208; T/L: 457 2208
> e-mail: reinwald@us.ibm.com
>
>
>
> From:   Shirish Tatikonda <shirish.tatikonda@gmail.com>
> To:     dev@systemml.incubator.apache.org
> Date:   12/16/2015 10:03 PM
> Subject:        Re: DML example on main SystemML website
>
>
>
> Deron,
>
> Along with such a complete algorithm, we could also include one/two common
> and useful DML snippets. We could also create a "DML Cookbook" with such
> snippets and keep adding more over time.
> Some example snippets are below -- note that I created them quite a while
> back, and they may need revision and testing.
>
> *Classifier Performance*
>
> # Confusion matrix
> cm = table(truthLabels, predictedLabels)
> TP = as.scalar(cm[1,1])
> TN = as.scalar(cm[2,2])
> FP = as.scalar(cm[1,2])
> FN = as.scalar(cm[2,1])
>
> accuracy = (TP+TN)/nrow(truthLabels)
> precision = TP / (TP+FP)
> recall = TP / (TP+FN)
>
> print("Accuracy = " + accuracy + ", Precision = " + precision + ", Recall
> =
> " + recall);
>
> *Covariance matrix*
>
> A = read("input.mtx");
> N = nrow(A);
> # column means
> mu = colSums(A)/N;
> # Covarianace matrix
> C = (t(A) %*% A)/(N-1) - (N/(N-1))*t(mu) %*% mu;
>
> *Select rows satisfying a predicate*
>
> ind = diag(ppred(A[,1], thresh, ">"));
> ind = removeEmpty(target=ind, margin="rows");
> result = ind %*% A;
>
>
> *Center and Scale columns*
>
> A = read("input.mtx");
> cm = colMeans(A);
> cvars = (colSums (A^2));
> cvars = (cvars - N*(cm^2))/(N-1);
> Ascaled = (A-cm)/sqrt(cvars);
>
>
> *Random shuffling of rows*
>
> N = nrow(A);
> s = sample(N, N, replace=FALSE);
> tab = table(seq(1:N), s);
> result = tab %*% A;
>
> On Wed, Dec 16, 2015 at 5:59 PM, Deron Eriksson <deroneriksson@gmail.com>
> wrote:
>
> > That example is perfect. Concise and powerful. Thank you Fred.
> >
> > Deron
> >
> >
> > On Wed, Dec 16, 2015 at 4:16 PM, Frederick R Reiss <frreiss@us.ibm.com>
> > wrote:
> >
> > > We can use the Poisson nonnegative matrix factorization example from
> last
> > > week's webcast:
> > >
> > >    i = 0
> > >    while(i < max_iterations) {
> > >        H = (H * (t(W) %*% (V/(W%*%H + epsilon)))) / t(colSums(W))
> > >        W = (W * ((V/(W%*%H) + epsilon) %*% t(H))) / t(rowSums(H))
> > >        i = i + 1;
> > >    }
> > >
> > >
> > > Sound ok to everyone?
> > >
> > > Fred
> > >
> > >
> > > [image: Inactive hide details for Deron Eriksson ---12/16/2015
> 04:02:26
> > > PM---Hi, I think the main SystemML website at http://systemml.i]Deron
> > > Eriksson ---12/16/2015 04:02:26 PM---Hi, I think the main SystemML
> > website
> > > at http://systemml.incubator.apache.org/
> > >
> > > From: Deron Eriksson <deroneriksson@gmail.com>
> > > To: dev@systemml.incubator.apache.org
> > > Date: 12/16/2015 04:02 PM
> > > Subject: DML example on main SystemML website
> > > ------------------------------
> > >
> > >
> > >
> > > Hi,
> > >
> > > I think the main SystemML website at
> > http://systemml.incubator.apache.org/
> > > needs to be updated so that the DML example is an actual algorithm or
> at
> > > least a fragment of an algorithm.
> > >
> > > Does anyone have a recommendation for a short, concise example that
> shows
> > > the power of DML?
> > >
> > > Thanks!
> > > Deron
> > >
> > >
> > >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message