lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@cs.put.poznan.pl>
Subject Re: Clustering Carrot2 vs TermVector Analysis
Date Wed, 01 Jun 2005 08:24:07 GMT

Hi Andrew,

Coming up with an answer... sorry for the delay.

> By using the carrot demo: 
> http://www.newsarch.com/archive/mailinglist/jakarta/lucene/user/msg03928.html
> 
> 
> I was able to easliy cluster search results based on the fields used
> by carrot( url, title, and summary). However I was wondering if there
> was a way to do something similar using term vector analysis and the
> built in TermVector / Similarity api.

Yes, most clustering methods are based just on that (term-vector
matrix). Carrot also uses this internally, but builds its own data
structure from the provided data instead of relying on Lucene's. It
shouldn't be a problem to write a clustering plugin to Carrot that
actually uses the term-vector data from Lucene.

> After doing a typical lucene search how can I get the  top 5 "key
> terms" for each of the top ten documents.  I was thinking that I sum
> these and then have a type of cluster.

The question is ill-defined, I'm afraid. "top 5 key terms" are very 
subjecting and depend on the strategy of score calculation, the way 
you're pruning stop words, etc.

I also don't get the: "each of the top ten documents". Do you mean: each 
of the ten top documents within a cluster?

D.

P.S. Please CC me directly; I read mails to newsgroups in batches every 
few days.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message