mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Quinn <squ...@gatech.edu>
Subject Re: Spectral clustering
Date Thu, 07 May 2015 22:50:19 GMT
Hi Sugam,

To clarify, the "RBF" I mentioned for computing affinities is the radial 
basis function, linked in mahout's spectral clustering documentation: 
http://en.wikipedia.org/wiki/RBF_kernel

The basic layout is to compare documents pairwise, use RBF to compute 
their similiarity, and set the entries in the affinity matrix 
corresponding to the two documents to the output of the RBF.

On 5/7/15 4:59 PM, Shannon Quinn wrote:
> Hi Sugam,
>
> This is in response to your original thread: 
> http://mail-archives.apache.org/mod_mbox/mahout-user/201505.mbox/%3C1412053714.1387791.1431020729309.JavaMail.yahoo%40mail.yahoo.com%3E
>
> The first thing you need to do is build the graph affinity matrix 
> yourself. That's the input to the map-reduce spectral clustering 
> algorithm, and what is described in the documentation (the "i, j, 
> value" part). Basically you'll consider each document as a single node 
> in a graph, and weight the connections between nodes. "i" and "j" are 
> the pair of nodes you're considering, and "value" is the similarity / 
> affinity, usually between 0 (completely dissimilar) and 1 (identical). 
> Typically you use RBF to compute affinities.
>
> Once you have the data in this format, then you can feed it to the 
> spectral clustering algorithm. Having the Mahout package compute the 
> affinities is at the top of my to-do list for the next version (though 
> there are still some questions that have to be addressed), so in 
> theory you could just submit the documents as you would to any other 
> algorithm in Mahout, but for now you have to compute the affinities 
> yourself.
>
> Let me know if anything still isn't clear.
>
> Shannon


Mime
View raw message