Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of iorixxx@yahoo.com designates
 98.138.91.249 as permitted sender)
References: 
 <CALY=cQBT3zazQpZ4MrkzbcXP_9XcWhDTiuyayO0L2rP2jD_sbA@mail.gmail.com>
Message-ID: <1402596102.28045.YahooMailNeo@web124703.mail.ne1.yahoo.com>
Date: Thu, 12 Jun 2014 11:01:42 -0700 (PDT)
From: Ahmet Arslan <iorixxx@yahoo.com.INVALID>
Reply-To: Ahmet Arslan <iorixxx@yahoo.com>
Subject: Re: Relevancy tests
To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>
In-Reply-To: 
 <CALY=cQBT3zazQpZ4MrkzbcXP_9XcWhDTiuyayO0L2rP2jD_sbA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Hi,

Relevance Judgments are labor intensive and expensive. Some Information Retrieval forums ( TREC, CLEF, etc) provide these golden sets. But they are not public.

http://rosenfeldmedia.com/books/search-analytics/ talks about how to create a "golden set" for your top n queries.


Also there are some works describing how to tune parameters of search system using click trough data.


On Thursday, June 12, 2014 8:47 PM, Ivan Brusic <ivan@brusic.com> wrote:
Perhaps more of an NLP question, but are there any tests regarding
relevance for Lucene? Given an example corpus of documents, what are the
golden sets for specific queries? The Wikidump dump is used as a
benchmarking tool for both indexing and querying in Lucene, but there are
no metrics in terms of precision.

The Open Relevance project was closed yesterday (
http://lucene.apache.org/openrelevance/), which is what prompted me to ask
this question. Was the sub-project closed because others have found
alternate solutions?

Relevancy is of course extremely context-dependent and objective, but my
hope is that there is an example catalog somewhere with defined golden sets.

Cheers,

Ivan


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org