lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: Lucene's Mean Average Precision
Date Sun, 04 May 2008 23:06:15 GMT
How did you arrive at that MAP?  What analyzers, etc.?  So much of  
search depends on your choices during indexing and querying, etc.   
There is some work by the IBM Haifa people up on the Wiki, so that  
would be one place to check.  Another question is what is your end  
goal is in doing a TREC evaluation?  Are you competing in TREC, or are  
you just trying to get a feel for how Lucene performs?  If it is the  
latter, TREC is just one data point in the relevance sea and you can  
definitely risk overtuning for TREC.  You will definitely need a means  
for evaluating your data set.

And yes, most evals. I have seen pay attention to things like: pseudo  
relevance feedback and other expansion techniques, query construction,  
as well as adjusting Lucene's length normalization (see the Similarity  
class.)  You might also have a look at the contrib/benchmark Quality  
package.  Finally, one of these days, I am hoping we will get some  
TREC collections for use by committers on ASF hardware.  This process  
has been started and hopefully will be available soon.

Cheers,
Grant

On May 4, 2008, at 2:13 PM, DanaWhite wrote:

>
> Hello all,
>
> I have been doing some evaluation of Lucene on a TReC collection and  
> get a
> rather disappointing mean average precision (MAP) of 11%.  Other  
> sources
> seem to report a MAP of about 20%.
>
> So I am here to ask all of you who have done some Lucene evaluation  
> - what
> did you find Lucene's MAP to be?  Did you do anything to special  
> (e.g. query
> expansion) to get it higher?
>
> Thanks
> Dana
> -- 
> View this message in context: http://www.nabble.com/Lucene%27s-Mean-Average-Precision-tp17049412p17049412.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>




--------------------------
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ







---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message