Thanks Jeff, I’ve found the root cause, the
input vectors contain some “Infinity” value, it’s because a bug in our logic to
generate tfidf value, we fixed the bug, now the KMeans process works as
expected, thank you.
BTW, we plan to use Canopy to find the
initial cluster centers, but the process seems it’s very slow, do you have any
good idea? Thanks
Cheers
Ramon
> Date: Fri, 16 Mar 2012 10:16:55 0600
> From: jdog@windwardsolutions.com
> To: user@mahout.apache.org
> Subject: Re: Why there is "Infinity" values for the vector of a KMeans cluster center
point?
>
> Good question. The only way I can think of an infinity in a Kluster
> center is if there were some infinity values in the vectors it observed.
> The center (centroid) is calculated in each iteration after all points
> have been observed by dividing S1 by S0. If, for some reason, S0 was
> zero this would cause all of the center elements to be infinity. We
> check for that case so it is unlikely.
>
> Can you narrow it down a bit more? How are you getting the kmeans prior?
> By sampling input vectors (k) or using Canopy? Are there any infinity
> values in clusters0?
>
>
> On 3/15/12 10:11 PM, WangRamon wrote:
> >
> >
> >
> >
> >
> > Hi Guys
> >
> >
> >
> > I’m running kMeans driver, I find most (95%)
> > of my input vectors is built into one cluster, so I retrieved the big Cluster
> > object and one of its point that belong to it, use CosineDistanceMeasure to calculate
> > the distance between them, I found the distance is “NaN”, so I tried to debug,
> > and found the center point vector of the Cluster object contains some “Infinity”
> > values, so when the measure method is called, the AbstractVector.dot will
> > return “Infinity”, so I don’t know if this is the reason that caused most
of my
> > input vectors belong to one big cluster? And why there are “Infinity” values
> > in the center point? Thanks in advance. BTW, i'm suing Mahout 0.6 release.
> >
> >
> >
> > Cheers
> >
> > Ramon
> >
> >
>
