lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <grant.ingers...@gmail.com>
Subject Re: search quality - assessment & improvements
Date Mon, 25 Jun 2007 18:48:03 GMT

On Jun 25, 2007, at 2:19 PM, Doron Cohen wrote:
>> IANAL and I didn't read the link, but I think people publish their
>> MAP scores, etc. all the time on TREC data.  I think it implies that
>> you obtained the data through legal means.
>
> So you're saying that if person "X" got the TREC data legally, we  
> can have
> in our (say) benchmarks age, something like:
>   (*) Person "X" reports the following TREC measures...
> And anyone discussing his TREC results with Lucene in Lucene's mailing
> lists does this under the list assumption that he got the TREC data
> legally. Sounds practical to me, at least to start with.

It seems reasonable, but I am not an authority.  One way to do it, is  
to look for TREC citations in papers.  By the way, the link in your  
orig. paper is password protected.  A search for TREC precision  
recall on Yahoo! yields papers that discuss past runs of TREC that  
were not published as part of TREC.  Of course, that doesn't make it  
right, but my gut feeling is it is not a big deal assuming you came  
about the data legally.  In general, people publish their precision  
and recall scores given a collection.   Without the name of the  
collection, the scores are meaningless.

-Grant


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message