Return-Path: Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: (qmail 36958 invoked from network); 25 Apr 2010 09:23:29 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 25 Apr 2010 09:23:29 -0000 Received: (qmail 20392 invoked by uid 500); 25 Apr 2010 09:23:28 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 20007 invoked by uid 500); 25 Apr 2010 09:23:28 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 20000 invoked by uid 99); 25 Apr 2010 09:23:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 25 Apr 2010 09:23:27 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=AWL,HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [85.25.71.29] (HELO mail.troja.net) (85.25.71.29) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 25 Apr 2010 09:23:21 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.troja.net (Postfix) with ESMTP id 88124D36005 for ; Sun, 25 Apr 2010 11:22:57 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mail.troja.net Received: from mail.troja.net ([127.0.0.1]) by localhost (megaira.troja.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3lUwGp4h2G+e for ; Sun, 25 Apr 2010 11:22:45 +0200 (CEST) Received: from VEGA (port-83-236-62-54.dynamic.qsc.de [83.236.62.54]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.troja.net (Postfix) with ESMTPSA id 1DB10D36002 for ; Sun, 25 Apr 2010 11:22:45 +0200 (CEST) From: "Uwe Schindler" To: References: <01b501cae454$66a84510$33f8cf30$@de> In-Reply-To: <01b501cae454$66a84510$33f8cf30$@de> Subject: RE: Small sorting problem in spatial contrib Date: Sun, 25 Apr 2010 11:22:57 +0200 Message-ID: <01bd01cae458$e4ed4390$aec7cab0$@de> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01BE_01CAE469.A8761390" X-Mailer: Microsoft Office Outlook 12.0 Thread-index: AcrkInz2TTvHDwjmQoS0LMlMSNXQaQAMbHtAAAEcUAA= Content-language: de ------=_NextPart_000_01BE_01CAE469.A8761390 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Here some links to issues, that show the problem. =20 >From you code it looks like you are still using 3.0.0 not 3.0.1 (as = CustomScoreQuery.customScore is deprecated and fixed in 3.0.1, as this = is the root of the problem): = https://issues.apache.org/jira/browse/LUCENE-2190 =20 The new DistanceQuery is planned here: https://issues.apache.org/jira/browse/LUCENE-2395 =20 Uwe =20 ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: uwe@thetaphi.de =20 From: Uwe Schindler [mailto:uwe@thetaphi.de]=20 Sent: Sunday, April 25, 2010 10:51 AM To: dev@lucene.apache.org Subject: RE: Small sorting problem in spatial contrib =20 If you look into the source code of this CustomScore Spatial Test, there = is mentioned that it is broken. The problem is that the spatial contrib. = sort does not work correct with multiple segments. We are working on = that, no solution here. We will have DistanceQuery in future, that does = not need sorting, as it will score with the distance in adition to = normal scoring. =20 ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de =20 eMail: uwe@thetaphi.de =20 From: Alex [mailto:azlist1@gmail.com]=20 Sent: Sunday, April 25, 2010 4:53 AM To: java-dev@lucene.apache.org Subject: Small sorting problem in spatial contrib =20 Hi,=20 I've been playing around with the SpatialFilter that Chris Male and = others ares developing.=20 I must say it seems to work fine and I'm really eager to see the final = release.=20 However I think I found a small bug that occurs at scoring time in some = situations. The SpatialFilter does its job pretty well but when TestSpatialFilter.DistanceCustomScoreQuery.customScore(int doc, float = subQueryScore, float valSrcScore) { Double docDistance =3D = spatialFilter.getDistanceFilter().getDistance(doc); is called I get=20 docDistance =3D null=20 which prevents the scoring to occur correctly.=20 I've been reading the code a litle to see what occurs under the hood and = it seems to me the problem is linked with the ThreadedDistanceFilter and = IndexSearcher.=20 The problem really occurs when IndexSearcher's public void search(Weight = weight, Filter filter, Collector collector) is called and when it has = more than 1 subReader=20 near line 260 of IndexSearcher for (int i =3D 0; i < subReaders.length; i++) { // search each subreader collector.setNextReader(subReaders[i], docStarts[i]); searchWithFilter(subReaders[i], weight, filter, collector); } What seems to happen is that ThreadedDistanceFilter stores the correct = document id in it's inner distances Map but somehow the value that is = passed to=20 TestSpatialFilter.DistanceCustomScoreQuery.customScore(int doc, float = subQueryScore, float valSrcScore) is not correct and probably corresponds to the bit position of the = subReader ... I hope I'm clear enough ...=20 :) I have no idea on how to fix this. I hope this helps you guys and you = can find a workaround.=20 Let me know what I can do to fix this problem. Thanks for helping.=20 Cheers, Alex ------=_NextPart_000_01BE_01CAE469.A8761390 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable

Here some links to issues, that show the = problem.

 

From you code it looks like you are still using 3.0.0 not = 3.0.1 (as CustomScoreQuery.customScore is deprecated and fixed in 3.0.1, as = this is the root of the problem): https://issues= .apache.org/jira/browse/LUCENE-2190

 

The new DistanceQuery is planned = here:

https://issues= .apache.org/jira/browse/LUCENE-2395

 

Uwe

 

-----

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 = Bremen

http://www.thetaphi.de<= /p>

eMail: uwe@thetaphi.de

 

From:= Uwe = Schindler [mailto:uwe@thetaphi.de]
Sent: Sunday, April 25, 2010 10:51 AM
To: dev@lucene.apache.org
Subject: RE: Small sorting problem in spatial = contrib

 

If you look into the source code of this CustomScore = Spatial Test, there is mentioned that it is broken. The problem is that the = spatial contrib. sort does not work correct with multiple segments. We are = working on that, no solution here. We will have DistanceQuery in future, that does = not need sorting, as it will score with the distance in adition to normal = scoring.

 

-----

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 = Bremen

http://www.thetaphi.de

eMail: uwe@thetaphi.de

 

From:= Alex [mailto:azlist1@gmail.com]
Sent: Sunday, April 25, 2010 4:53 AM
To: java-dev@lucene.apache.org
Subject: Small sorting problem in spatial = contrib

 

Hi,

I've been playing around with the SpatialFilter that Chris Male and = others ares developing.
I must say it seems to work fine and I'm really eager to see the final = release.

However I think I found a small bug  that occurs at scoring time in = some situations.

The SpatialFilter does its job pretty well but when

TestSpatialFilter.DistanceCustomScoreQuery.customScore(int doc, float subQueryScore, float valSrcScore) {
Double docDistance =3D = spatialFilter.getDistanceFilter().getDistance(doc);

 is called I get

docDistance =3D null

which prevents the scoring to occur correctly.

I've been reading the code a litle to see what occurs under the hood and = it seems to me the problem is linked with the ThreadedDistanceFilter and IndexSearcher.

The problem really occurs when IndexSearcher's public void search(Weight weight, Filter filter, Collector collector) is called and when it has = more than 1 subReader

near line 260 of IndexSearcher

for (int i =3D 0; i < subReaders.length; i++) { // search each = subreader
        = collector.setNextReader(subReaders[i], docStarts[i]);
        = searchWithFilter(subReaders[i], weight, filter, collector);
}



What seems to happen is that ThreadedDistanceFilter stores the correct = document id in it's inner distances Map but somehow the value that is passed to =
TestSpatialFilter.DistanceCustomScoreQuery.customScore(int doc, float subQueryScore, float valSrcScore)
is not correct and probably corresponds to the bit position of the = subReader ...

I hope I'm clear enough ...
:)

I have no idea on how to fix this. I hope this helps you guys and you = can find a workaround.
Let me know what I can do to fix this problem.


Thanks for helping.

Cheers,


Alex

------=_NextPart_000_01BE_01CAE469.A8761390--