Hi Sugam,
To clarify, the "RBF" I mentioned for computing affinities is the radial
basis function, linked in mahout's spectral clustering documentation:
http://en.wikipedia.org/wiki/RBF_kernel
The basic layout is to compare documents pairwise, use RBF to compute
their similiarity, and set the entries in the affinity matrix
corresponding to the two documents to the output of the RBF.
On 5/7/15 4:59 PM, Shannon Quinn wrote:
> Hi Sugam,
>
> This is in response to your original thread:
> http://mailarchives.apache.org/mod_mbox/mahoutuser/201505.mbox/%3C1412053714.1387791.1431020729309.JavaMail.yahoo%40mail.yahoo.com%3E
>
> The first thing you need to do is build the graph affinity matrix
> yourself. That's the input to the mapreduce spectral clustering
> algorithm, and what is described in the documentation (the "i, j,
> value" part). Basically you'll consider each document as a single node
> in a graph, and weight the connections between nodes. "i" and "j" are
> the pair of nodes you're considering, and "value" is the similarity /
> affinity, usually between 0 (completely dissimilar) and 1 (identical).
> Typically you use RBF to compute affinities.
>
> Once you have the data in this format, then you can feed it to the
> spectral clustering algorithm. Having the Mahout package compute the
> affinities is at the top of my todo list for the next version (though
> there are still some questions that have to be addressed), so in
> theory you could just submit the documents as you would to any other
> algorithm in Mahout, but for now you have to compute the affinities
> yourself.
>
> Let me know if anything still isn't clear.
>
> Shannon
