lucene-java-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dor...@apache.org
Subject svn commit: r560372 [2/2] - in /lucene/java/trunk: ./ contrib/benchmark/ contrib/benchmark/src/java/org/apache/lucene/benchmark/quality/ contrib/benchmark/src/java/org/apache/lucene/benchmark/quality/trec/ contrib/benchmark/src/java/org/apache/lucene/b...
Date Fri, 27 Jul 2007 20:24:53 GMT
Added: lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecQRels.txt
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecQRels.txt?view=auto&rev=560372
==============================================================================
--- lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecQRels.txt
(added)
+++ lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecQRels.txt
Fri Jul 27 13:24:52 2007
@@ -0,0 +1,723 @@
+# -----------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# -----------------------------------------------------------------------
+
+# ------------------------------------------------------------
+# Format:
+#
+#       qnum   0   doc-name     is-relevant
+#
+#
+# The origin of this file was created using 
+# utils.QualityQueriesFinder, so all queries 
+# would have perfect 1.0 for all meassures.
+#
+# To make it suitable for testing it was modified
+# for some queries, depending on m = qnum % 8
+# m==0: avg_precision and recall are hurt, by marking fake docs as relevant
+# m==1: precision_at_n and avg_precision are hurt, by unmarking relevant docs
+# m==2: all precision, precision_at_n and recall are hurt.
+# m>=3: these queries remain perfect
+# ------------------------------------------------------------
+
+# --- m==0: avg_precision and recall are hurt, by marking fake docs as relevant
+
+0 	 0 	 fakedoc1             	 1
+0 	 0 	 fakedoc2             	 1
+0 	 0 	 fakedoc3             	 1
+0 	 0 	 fakedoc4             	 1
+
+0 	 0 	 doc18211             	 1
+0 	 0 	 doc20192             	 1
+0 	 0 	 doc7401              	 1
+0 	 0 	 doc11285             	 1
+0 	 0 	 doc20647             	 1
+0 	 0 	 doc3057              	 1
+0 	 0 	 doc12431             	 1
+0 	 0 	 doc4989              	 1
+0 	 0 	 doc17324             	 1
+0 	 0 	 doc4030              	 1
+0 	 0 	 doc4290              	 1
+0 	 0 	 doc3462              	 1
+0 	 0 	 doc15313             	 1
+0 	 0 	 doc10303             	 1
+0 	 0 	 doc1893              	 1
+0 	 0 	 doc5008              	 1
+0 	 0 	 doc14634             	 1
+0 	 0 	 doc5471              	 1
+0 	 0 	 doc17904             	 1
+0 	 0 	 doc7168              	 1
+0 	 0 	 doc21275             	 1
+0 	 0 	 doc9011              	 1
+0 	 0 	 doc17546             	 1
+0 	 0 	 doc9102              	 1
+0 	 0 	 doc13199             	 1
+
+# --- m==1: precision_at_n and avg_precision are hurt, by unmarking relevant docs
+
+1 	 0 	 doc9857              	 0
+1 	 0 	 doc16846             	 1
+1 	 0 	 doc4320              	 1
+1 	 0 	 doc9501              	 0
+1 	 0 	 doc10159             	 1
+1 	 0 	 doc16642             	 1
+1 	 0 	 doc17536             	 0
+1 	 0 	 doc17571             	 1
+1 	 0 	 doc18728             	 1
+1 	 0 	 doc18828             	 1
+1 	 0 	 doc19108             	 0
+1 	 0 	 doc9940              	 1
+1 	 0 	 doc11852             	 1
+1 	 0 	 doc7430              	 0
+1 	 0 	 doc19162             	 1
+1 	 0 	 doc1743              	 1
+1 	 0 	 doc2137              	 1
+1 	 0 	 doc7611              	 1
+1 	 0 	 doc8072              	 1
+1 	 0 	 doc12764             	 1
+1 	 0 	 doc2593              	 1
+1 	 0 	 doc11088             	 1
+1 	 0 	 doc931               	 1
+1 	 0 	 doc7673              	 1
+1 	 0 	 doc12941             	 1
+1 	 0 	 doc11797             	 1
+1 	 0 	 doc11831             	 1
+1 	 0 	 doc13162             	 1
+1 	 0 	 doc4423              	 1
+1 	 0 	 doc5217              	 1
+
+# ---- m==2: all precision, precision_at_n and recall are hurt.
+
+2 	 0 	 fakedoc1             	 1
+2 	 0 	 fakedoc2             	 1
+2 	 0 	 fakedoc3             	 1
+2 	 0 	 fakedoc4             	 1
+
+2 	 0 	 doc3137              	 0
+2 	 0 	 doc7142              	 0
+2 	 0 	 doc13667             	 0
+2 	 0 	 doc13171             	 0
+2 	 0 	 doc13372             	 1
+2 	 0 	 doc21415             	 1
+2 	 0 	 doc16298             	 1
+2 	 0 	 doc14957             	 1
+2 	 0 	 doc153               	 1
+2 	 0 	 doc16092             	 1
+2 	 0 	 doc16096             	 1
+2 	 0 	 doc21303             	 1
+2 	 0 	 doc18681             	 1
+2 	 0 	 doc20756             	 1
+2 	 0 	 doc355               	 1
+2 	 0 	 doc13395             	 1
+2 	 0 	 doc5009              	 1
+2 	 0 	 doc17164             	 1
+2 	 0 	 doc13162             	 1
+2 	 0 	 doc11757             	 1
+2 	 0 	 doc9637              	 1
+2 	 0 	 doc18087             	 1
+2 	 0 	 doc4593              	 1
+2 	 0 	 doc4677              	 1
+2 	 0 	 doc20865             	 1
+2 	 0 	 doc8556              	 1
+2 	 0 	 doc2578              	 1
+2 	 0 	 doc1163              	 1
+2 	 0 	 doc3797              	 1
+2 	 0 	 doc11094             	 1
+
+
+3 	 0 	 doc19578             	 1
+3 	 0 	 doc14860             	 1
+3 	 0 	 doc7235              	 1
+3 	 0 	 doc20590             	 1
+3 	 0 	 doc17933             	 1
+3 	 0 	 doc9384              	 1
+3 	 0 	 doc10783             	 1
+3 	 0 	 doc1963              	 1
+3 	 0 	 doc18356             	 1
+3 	 0 	 doc13254             	 1
+3 	 0 	 doc18402             	 1
+3 	 0 	 doc15241             	 1
+3 	 0 	 doc3303              	 1
+3 	 0 	 doc8868              	 1
+3 	 0 	 doc18520             	 1
+3 	 0 	 doc4650              	 1
+3 	 0 	 doc4727              	 1
+3 	 0 	 doc21518             	 1
+3 	 0 	 doc5060              	 1
+3 	 0 	 doc7587              	 1
+3 	 0 	 doc2990              	 1
+3 	 0 	 doc8042              	 1
+3 	 0 	 doc6304              	 1
+3 	 0 	 doc13223             	 1
+3 	 0 	 doc1964              	 1
+3 	 0 	 doc10597             	 1
+3 	 0 	 doc21023             	 1
+3 	 0 	 doc19057             	 1
+3 	 0 	 doc14948             	 1
+3 	 0 	 doc9692              	 1
+
+
+4 	 0 	 doc2534              	 1
+4 	 0 	 doc21388             	 1
+4 	 0 	 doc20923             	 1
+4 	 0 	 doc11547             	 1
+4 	 0 	 doc19755             	 1
+4 	 0 	 doc3793              	 1
+4 	 0 	 doc6714              	 1
+4 	 0 	 doc12722             	 1
+4 	 0 	 doc5552              	 1
+4 	 0 	 doc6810              	 1
+4 	 0 	 doc16953             	 1
+4 	 0 	 doc2527              	 1
+4 	 0 	 doc5361              	 1
+4 	 0 	 doc12353             	 1
+4 	 0 	 doc7308              	 1
+4 	 0 	 doc3836              	 1
+4 	 0 	 doc2293              	 1
+4 	 0 	 doc7348              	 1
+4 	 0 	 doc17119             	 1
+4 	 0 	 doc19331             	 1
+4 	 0 	 doc3411              	 1
+4 	 0 	 doc14643             	 1
+4 	 0 	 doc9058              	 1
+4 	 0 	 doc11099             	 1
+4 	 0 	 doc12485             	 1
+4 	 0 	 doc16432             	 1
+4 	 0 	 doc10047             	 1
+4 	 0 	 doc13788             	 1
+4 	 0 	 doc117               	 1
+4 	 0 	 doc638               	 1
+
+
+
+5 	 0 	 doc169               	 1
+5 	 0 	 doc13181             	 1
+5 	 0 	 doc4350              	 1
+5 	 0 	 doc10242             	 1
+5 	 0 	 doc955               	 1
+5 	 0 	 doc5389              	 1
+5 	 0 	 doc17122             	 1
+5 	 0 	 doc17417             	 1
+5 	 0 	 doc12199             	 1
+5 	 0 	 doc6918              	 1
+5 	 0 	 doc3857              	 1
+5 	 0 	 doc2981              	 1
+5 	 0 	 doc10639             	 1
+5 	 0 	 doc10478             	 1
+5 	 0 	 doc8573              	 1
+5 	 0 	 doc9197              	 1
+5 	 0 	 doc9298              	 1
+5 	 0 	 doc2492              	 1
+5 	 0 	 doc10262             	 1
+5 	 0 	 doc5180              	 1
+5 	 0 	 doc11758             	 1
+5 	 0 	 doc4065              	 1
+5 	 0 	 doc9124              	 1
+5 	 0 	 doc11528             	 1
+5 	 0 	 doc18879             	 1
+5 	 0 	 doc17864             	 1
+5 	 0 	 doc3204              	 1
+5 	 0 	 doc12157             	 1
+5 	 0 	 doc4496              	 1
+5 	 0 	 doc20190             	 1
+
+
+
+6 	 0 	 doc9507              	 1
+6 	 0 	 doc15630             	 1
+6 	 0 	 doc8469              	 1
+6 	 0 	 doc11918             	 1
+6 	 0 	 doc20482             	 1
+6 	 0 	 doc20158             	 1
+6 	 0 	 doc19831             	 1
+6 	 0 	 doc8296              	 1
+6 	 0 	 doc8930              	 1
+6 	 0 	 doc16460             	 1
+6 	 0 	 doc2577              	 1
+6 	 0 	 doc15476             	 1
+6 	 0 	 doc1767              	 1
+6 	 0 	 doc689               	 1
+6 	 0 	 doc16606             	 1
+6 	 0 	 doc6149              	 1
+6 	 0 	 doc18691             	 1
+6 	 0 	 doc2208              	 1
+6 	 0 	 doc3592              	 1
+6 	 0 	 doc11199             	 1
+6 	 0 	 doc16329             	 1
+6 	 0 	 doc6007              	 1
+6 	 0 	 doc15231             	 1
+6 	 0 	 doc20622             	 1
+6 	 0 	 doc21468             	 1
+6 	 0 	 doc12230             	 1
+6 	 0 	 doc5723              	 1
+6 	 0 	 doc8120              	 1
+6 	 0 	 doc8668              	 1
+6 	 0 	 doc303               	 1
+
+
+
+
+7 	 0 	 doc7728              	 1
+7 	 0 	 doc7693              	 1
+7 	 0 	 doc21088             	 1
+7 	 0 	 doc5017              	 1
+7 	 0 	 doc10807             	 1
+7 	 0 	 doc16204             	 1
+7 	 0 	 doc2233              	 1
+7 	 0 	 doc3632              	 1
+7 	 0 	 doc4719              	 1
+7 	 0 	 doc6477              	 1
+7 	 0 	 doc6502              	 1
+7 	 0 	 doc6709              	 1
+7 	 0 	 doc7710              	 1
+7 	 0 	 doc9193              	 1
+7 	 0 	 doc9309              	 1
+7 	 0 	 doc9789              	 1
+7 	 0 	 doc10971             	 1
+7 	 0 	 doc18059             	 1
+7 	 0 	 doc19906             	 1
+7 	 0 	 doc20089             	 1
+7 	 0 	 doc20102             	 1
+7 	 0 	 doc21040             	 1
+7 	 0 	 doc21153             	 1
+7 	 0 	 doc9147              	 1
+7 	 0 	 doc9930              	 1
+7 	 0 	 doc19763             	 1
+7 	 0 	 doc1559              	 1
+7 	 0 	 doc21248             	 1
+7 	 0 	 doc17945             	 1
+7 	 0 	 doc526               	 1
+
+
+# --- m==0: avg_precision and recall are hurt, by marking fake docs as relevant
+
+8 	 0 	 fakedoc1             	 1
+8 	 0 	 fakedoc2             	 1
+8 	 0 	 fakedoc3             	 1
+8 	 0 	 fakedoc4             	 1
+
+8 	 0 	 doc16299             	 1
+8 	 0 	 doc1662              	 1
+8 	 0 	 doc4585              	 1
+8 	 0 	 doc12315             	 1
+8 	 0 	 doc16266             	 1
+8 	 0 	 doc13136             	 1
+8 	 0 	 doc19212             	 1
+8 	 0 	 doc7086              	 1
+8 	 0 	 doc7062              	 1
+8 	 0 	 doc6134              	 1
+8 	 0 	 doc13953             	 1
+8 	 0 	 doc16264             	 1
+8 	 0 	 doc2494              	 1
+8 	 0 	 doc10636             	 1
+8 	 0 	 doc10894             	 1
+8 	 0 	 doc6844              	 1
+8 	 0 	 doc674               	 1
+8 	 0 	 doc13520             	 1
+8 	 0 	 doc344               	 1
+8 	 0 	 doc2896              	 1
+8 	 0 	 doc11871             	 1
+8 	 0 	 doc1862              	 1
+8 	 0 	 doc16728             	 1
+8 	 0 	 doc10308             	 1
+8 	 0 	 doc2227              	 1
+8 	 0 	 doc13167             	 1
+8 	 0 	 doc20607             	 1
+8 	 0 	 doc9670              	 1
+8 	 0 	 doc1566              	 1
+8 	 0 	 doc17885             	 1
+
+
+# ---- m==1: precision_at_n and avg_precision are hurt, by unmarking relevant docs
+
+
+9 	 0 	 doc1990              	 0
+9 	 0 	 doc9342              	 1
+9 	 0 	 doc19427             	 1
+9 	 0 	 doc12432             	 0
+9 	 0 	 doc13480             	 1
+9 	 0 	 doc3322              	 1
+9 	 0 	 doc16044             	 1
+9 	 0 	 doc266               	 0
+9 	 0 	 doc3437              	 1
+9 	 0 	 doc5370              	 1
+9 	 0 	 doc10314             	 1
+9 	 0 	 doc4892              	 1
+9 	 0 	 doc5763              	 0
+9 	 0 	 doc14045             	 1
+9 	 0 	 doc1090              	 1
+9 	 0 	 doc7437              	 1
+9 	 0 	 doc5822              	 1
+9 	 0 	 doc4285              	 1
+9 	 0 	 doc17119             	 1
+9 	 0 	 doc21001             	 1
+9 	 0 	 doc4337              	 1
+9 	 0 	 doc5967              	 1
+9 	 0 	 doc10214             	 1
+9 	 0 	 doc12001             	 1
+9 	 0 	 doc18553             	 1
+9 	 0 	 doc12116             	 1
+9 	 0 	 doc5064              	 1
+9 	 0 	 doc5018              	 1
+9 	 0 	 doc5037              	 1
+9 	 0 	 doc8025              	 1
+
+
+# ---- m==2: all precision, precision_at_n and recall are hurt.
+
+10 	 0 	 fakedoc1             	 1
+10 	 0 	 fakedoc2             	 1
+10 	 0 	 fakedoc3             	 1
+10 	 0 	 fakedoc4             	 1
+
+10 	 0 	 doc17218             	 0
+10 	 0 	 doc10270             	 0
+10 	 0 	 doc5958              	 0
+10 	 0 	 doc19943             	 0
+10 	 0 	 doc6510              	 1
+10 	 0 	 doc16087             	 1
+10 	 0 	 doc14893             	 1
+10 	 0 	 doc8933              	 1
+10 	 0 	 doc4354              	 1
+10 	 0 	 doc16729             	 1
+10 	 0 	 doc16761             	 1
+10 	 0 	 doc6964              	 1
+10 	 0 	 doc16743             	 1
+10 	 0 	 doc7357              	 1
+10 	 0 	 doc2534              	 1
+10 	 0 	 doc18321             	 1
+10 	 0 	 doc18497             	 1
+10 	 0 	 doc11214             	 1
+10 	 0 	 doc11819             	 1
+10 	 0 	 doc10818             	 1
+10 	 0 	 doc15769             	 1
+10 	 0 	 doc5348              	 1
+10 	 0 	 doc14948             	 1
+10 	 0 	 doc7891              	 1
+10 	 0 	 doc9897              	 1
+10 	 0 	 doc15559             	 1
+10 	 0 	 doc14935             	 1
+10 	 0 	 doc14954             	 1
+10 	 0 	 doc6621              	 1
+10 	 0 	 doc6930              	 1
+
+
+11 	 0 	 doc11943             	 1
+11 	 0 	 doc286               	 1
+11 	 0 	 doc1574              	 1
+11 	 0 	 doc17916             	 1
+11 	 0 	 doc17918             	 1
+11 	 0 	 doc19213             	 1
+11 	 0 	 doc9337              	 1
+11 	 0 	 doc8593              	 1
+11 	 0 	 doc8800              	 1
+11 	 0 	 doc18580             	 1
+11 	 0 	 doc209               	 1
+11 	 0 	 doc1893              	 1
+11 	 0 	 doc11189             	 1
+11 	 0 	 doc17702             	 1
+11 	 0 	 doc10180             	 1
+11 	 0 	 doc11869             	 1
+11 	 0 	 doc9705              	 1
+11 	 0 	 doc8715              	 1
+11 	 0 	 doc12753             	 1
+11 	 0 	 doc10195             	 1
+11 	 0 	 doc3552              	 1
+11 	 0 	 doc16030             	 1
+11 	 0 	 doc4623              	 1
+11 	 0 	 doc3188              	 1
+11 	 0 	 doc8735              	 1
+11 	 0 	 doc151               	 1
+11 	 0 	 doc5792              	 1
+11 	 0 	 doc5194              	 1
+11 	 0 	 doc3393              	 1
+11 	 0 	 doc19027             	 1
+
+
+
+12 	 0 	 doc18198             	 1
+12 	 0 	 doc2444              	 1
+12 	 0 	 doc4305              	 1
+12 	 0 	 doc6544              	 1
+12 	 0 	 doc11639             	 1
+12 	 0 	 doc10640             	 1
+12 	 0 	 doc12192             	 1
+12 	 0 	 doc128               	 1
+12 	 0 	 doc10760             	 1
+12 	 0 	 doc10881             	 1
+12 	 0 	 doc2698              	 1
+12 	 0 	 doc3552              	 1
+12 	 0 	 doc20524             	 1
+12 	 0 	 doc1884              	 1
+12 	 0 	 doc9187              	 1
+12 	 0 	 doc3131              	 1
+12 	 0 	 doc2911              	 1
+12 	 0 	 doc2589              	 1
+12 	 0 	 doc3747              	 1
+12 	 0 	 doc3813              	 1
+12 	 0 	 doc5222              	 1
+12 	 0 	 doc6023              	 1
+12 	 0 	 doc6624              	 1
+12 	 0 	 doc7655              	 1
+12 	 0 	 doc9205              	 1
+12 	 0 	 doc12062             	 1
+12 	 0 	 doc15504             	 1
+12 	 0 	 doc13625             	 1
+12 	 0 	 doc18704             	 1
+12 	 0 	 doc2277              	 1
+
+
+
+13 	 0 	 doc4948              	 1
+13 	 0 	 doc21565             	 1
+13 	 0 	 doc17135             	 1
+13 	 0 	 doc1866              	 1
+13 	 0 	 doc13989             	 1
+13 	 0 	 doc5605              	 1
+13 	 0 	 doc13431             	 1
+13 	 0 	 doc2100              	 1
+13 	 0 	 doc16347             	 1
+13 	 0 	 doc16894             	 1
+13 	 0 	 doc6764              	 1
+13 	 0 	 doc8554              	 1
+13 	 0 	 doc8695              	 1
+13 	 0 	 doc8977              	 1
+13 	 0 	 doc19478             	 1
+13 	 0 	 doc14595             	 1
+13 	 0 	 doc2408              	 1
+13 	 0 	 doc2592              	 1
+13 	 0 	 doc10947             	 1
+13 	 0 	 doc15794             	 1
+13 	 0 	 doc5236              	 1
+13 	 0 	 doc14847             	 1
+13 	 0 	 doc3980              	 1
+13 	 0 	 doc1844              	 1
+13 	 0 	 doc42                	 1
+13 	 0 	 doc7783              	 1
+13 	 0 	 doc4557              	 1
+13 	 0 	 doc16423             	 1
+13 	 0 	 doc17170             	 1
+13 	 0 	 doc5822              	 1
+
+
+
+14 	 0 	 doc17172             	 1
+14 	 0 	 doc17210             	 1
+14 	 0 	 doc5044              	 1
+14 	 0 	 doc4627              	 1
+14 	 0 	 doc4683              	 1
+14 	 0 	 doc15126             	 1
+14 	 0 	 doc4538              	 1
+14 	 0 	 doc273               	 1
+14 	 0 	 doc19585             	 1
+14 	 0 	 doc16078             	 1
+14 	 0 	 doc4529              	 1
+14 	 0 	 doc4186              	 1
+14 	 0 	 doc12961             	 1
+14 	 0 	 doc19217             	 1
+14 	 0 	 doc5670              	 1
+14 	 0 	 doc1699              	 1
+14 	 0 	 doc4716              	 1
+14 	 0 	 doc12644             	 1
+14 	 0 	 doc18387             	 1
+14 	 0 	 doc336               	 1
+14 	 0 	 doc16130             	 1
+14 	 0 	 doc18718             	 1
+14 	 0 	 doc12527             	 1
+14 	 0 	 doc11797             	 1
+14 	 0 	 doc11831             	 1
+14 	 0 	 doc7538              	 1
+14 	 0 	 doc17259             	 1
+14 	 0 	 doc18724             	 1
+14 	 0 	 doc19330             	 1
+14 	 0 	 doc19206             	 1
+
+
+
+15 	 0 	 doc12198             	 1
+15 	 0 	 doc20371             	 1
+15 	 0 	 doc2947              	 1
+15 	 0 	 doc10750             	 1
+15 	 0 	 doc7239              	 1
+15 	 0 	 doc14189             	 1
+15 	 0 	 doc19474             	 1
+15 	 0 	 doc14776             	 1
+15 	 0 	 doc21270             	 1
+15 	 0 	 doc6387              	 1
+15 	 0 	 doc12908             	 1
+15 	 0 	 doc9573              	 1
+15 	 0 	 doc17102             	 1
+15 	 0 	 doc21482             	 1
+15 	 0 	 doc6524              	 1
+15 	 0 	 doc18034             	 1
+15 	 0 	 doc1358              	 1
+15 	 0 	 doc13147             	 1
+15 	 0 	 doc17731             	 1
+15 	 0 	 doc12890             	 1
+15 	 0 	 doc20887             	 1
+15 	 0 	 doc19508             	 1
+15 	 0 	 doc18498             	 1
+15 	 0 	 doc20642             	 1
+15 	 0 	 doc19878             	 1
+15 	 0 	 doc6556              	 1
+15 	 0 	 doc10272             	 1
+15 	 0 	 doc5720              	 1
+15 	 0 	 doc17578             	 1
+15 	 0 	 doc17164             	 1
+
+
+# --- m==0: avg_precision and recall are hurt, by marking fake docs as relevant
+
+16 	 0 	 fakedoc1             	 1
+16 	 0 	 fakedoc2             	 1
+16 	 0 	 fakedoc3             	 1
+16 	 0 	 fakedoc4             	 1
+
+16 	 0 	 doc4043              	 1
+16 	 0 	 doc14985             	 1
+16 	 0 	 doc15370             	 1
+16 	 0 	 doc15426             	 1
+16 	 0 	 doc1702              	 1
+16 	 0 	 doc3062              	 1
+16 	 0 	 doc16134             	 1
+16 	 0 	 doc15037             	 1
+16 	 0 	 doc8224              	 1
+16 	 0 	 doc5044              	 1
+16 	 0 	 doc8545              	 1
+16 	 0 	 doc7228              	 1
+16 	 0 	 doc12686             	 1
+16 	 0 	 doc16609             	 1
+16 	 0 	 doc13161             	 1
+16 	 0 	 doc3446              	 1
+16 	 0 	 doc16493             	 1
+16 	 0 	 doc19297             	 1
+16 	 0 	 doc13619             	 1
+16 	 0 	 doc3281              	 1
+16 	 0 	 doc15499             	 1
+16 	 0 	 doc7373              	 1
+16 	 0 	 doc9064              	 1
+16 	 0 	 doc1710              	 1
+16 	 0 	 doc15411             	 1
+16 	 0 	 doc10890             	 1
+16 	 0 	 doc3166              	 1
+16 	 0 	 doc17894             	 1
+16 	 0 	 doc4560              	 1
+16 	 0 	 doc12766             	 1
+
+
+# --- m==1: precision_at_n and avg_precision are hurt, by unmarking relevant docs
+
+17 	 0 	 doc3117              	 0
+17 	 0 	 doc7477              	 0
+17 	 0 	 doc7569              	 0
+17 	 0 	 doc20667             	 0
+17 	 0 	 doc20260             	 1
+17 	 0 	 doc17355             	 1
+17 	 0 	 doc11021             	 1
+17 	 0 	 doc20934             	 1
+17 	 0 	 doc552               	 1
+17 	 0 	 doc20856             	 1
+17 	 0 	 doc3524              	 1
+17 	 0 	 doc17343             	 1
+17 	 0 	 doc21055             	 1
+17 	 0 	 doc19032             	 1
+17 	 0 	 doc19786             	 1
+17 	 0 	 doc9281              	 1
+17 	 0 	 doc1695              	 1
+17 	 0 	 doc15940             	 1
+17 	 0 	 doc9215              	 1
+17 	 0 	 doc8335              	 1
+17 	 0 	 doc20936             	 1
+17 	 0 	 doc6914              	 1
+17 	 0 	 doc12122             	 1
+17 	 0 	 doc6618              	 1
+17 	 0 	 doc5049              	 1
+17 	 0 	 doc450               	 1
+17 	 0 	 doc19206             	 1
+17 	 0 	 doc18823             	 1
+17 	 0 	 doc5307              	 1
+17 	 0 	 doc17295             	 1
+
+
+# ---- m==2: all precision, precision_at_n and recall are hurt.
+
+18 	 0 	 fakedoc1             	 1
+18 	 0 	 fakedoc2             	 1
+18 	 0 	 fakedoc3             	 1
+18 	 0 	 fakedoc4             	 1
+
+18 	 0 	 doc8064              	 0
+18 	 0 	 doc18142             	 0
+18 	 0 	 doc19383             	 0
+18 	 0 	 doc21151             	 0
+18 	 0 	 doc4665              	 1
+18 	 0 	 doc2897              	 1
+18 	 0 	 doc6878              	 1
+18 	 0 	 doc14507             	 1
+18 	 0 	 doc2976              	 1
+18 	 0 	 doc11757             	 1
+18 	 0 	 doc12625             	 1
+18 	 0 	 doc14908             	 1
+18 	 0 	 doc12790             	 1
+18 	 0 	 doc17915             	 1
+18 	 0 	 doc11804             	 1
+18 	 0 	 doc12935             	 1
+18 	 0 	 doc8225              	 1
+18 	 0 	 doc18011             	 1
+18 	 0 	 doc10493             	 1
+18 	 0 	 doc17922             	 1
+18 	 0 	 doc1902              	 1
+18 	 0 	 doc14049             	 1
+18 	 0 	 doc1334              	 1
+18 	 0 	 doc1168              	 1
+18 	 0 	 doc4859              	 1
+18 	 0 	 doc7124              	 1
+18 	 0 	 doc9692              	 1
+18 	 0 	 doc18402             	 1
+18 	 0 	 doc9089              	 1
+18 	 0 	 doc15375             	 1
+
+
+19 	 0 	 doc5267              	 1
+19 	 0 	 doc2310              	 1
+19 	 0 	 doc11435             	 1
+19 	 0 	 doc15666             	 1
+19 	 0 	 doc12733             	 1
+19 	 0 	 doc7925              	 1
+19 	 0 	 doc2444              	 1
+19 	 0 	 doc4900              	 1
+19 	 0 	 doc10803             	 1
+19 	 0 	 doc8869              	 1
+19 	 0 	 doc5051              	 1
+19 	 0 	 doc9163              	 1
+19 	 0 	 doc529               	 1
+19 	 0 	 doc19546             	 1
+19 	 0 	 doc18561             	 1
+19 	 0 	 doc10634             	 1
+19 	 0 	 doc3979              	 1
+19 	 0 	 doc8833              	 1
+19 	 0 	 doc7652              	 1
+19 	 0 	 doc4804              	 1
+19 	 0 	 doc12616             	 1
+19 	 0 	 doc8419              	 1
+19 	 0 	 doc9431              	 1
+19 	 0 	 doc16235             	 1
+19 	 0 	 doc732               	 1
+19 	 0 	 doc2515              	 1
+19 	 0 	 doc7194              	 1
+19 	 0 	 doc16301             	 1
+19 	 0 	 doc4494              	 1
+19 	 0 	 doc4496              	 1

Propchange: lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecQRels.txt
------------------------------------------------------------------------------
    svn:eol-style = native

Propchange: lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecQRels.txt
------------------------------------------------------------------------------
    svn:executable = *

Added: lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecTopics.txt
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecTopics.txt?view=auto&rev=560372
==============================================================================
--- lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecTopics.txt
(added)
+++ lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecTopics.txt
Fri Jul 27 13:24:52 2007
@@ -0,0 +1,281 @@
+# -----------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# -----------------------------------------------------------------------
+
+# ------------------------------------------------------------
+# This file was created using utils.QualityQueriesFinder.
+# See also TrecQRels.txt.
+# ------------------------------------------------------------
+
+<top>
+<num> Number: 0
+
+<title> statement months  total 1987
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 1
+
+<title> agreed 15  against five
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 2
+
+<title> nine only  month international
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 3
+
+<title> finance any  10 government
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 4
+
+<title> issue next  years all
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 5
+
+<title> who major  ltd today
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 6
+
+<title> business revs  securities per
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 7
+
+<title> quarter time  note sales
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 8
+
+<title> february earlier  loss group
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 9
+
+<title> out end  made some
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 10
+
+<title> spokesman financial  30 expected
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 11
+
+<title> 1985 now  prices due
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 12
+
+<title> before board  record could
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 13
+
+<title> pay debt  because trade
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 14
+
+<title> meeting increase  four price
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 15
+
+<title> chairman rate  six interest
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 16
+
+<title> since current  between agreement
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 17
+
+<title> oil we  when president
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 18
+
+<title> capital through  foreign added
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>
+
+<top>
+<num> Number: 19
+
+<title> 20 while  common week
+
+<desc> Description:
+
+
+<narr> Narrative:
+
+
+</top>

Propchange: lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecTopics.txt
------------------------------------------------------------------------------
    svn:eol-style = native

Propchange: lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/quality/trecTopics.txt
------------------------------------------------------------------------------
    svn:executable = *



Mime
View raw message