Return-Path: 
 <solr-user-return-40561-apmail-lucene-solr-user-archive=lucene.apache.org@lucene.apache.org>
Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org
Received: (qmail 58999 invoked from network); 3 Sep 2010 15:03:21 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 3 Sep 2010 15:03:21 -0000
Received: (qmail 21849 invoked by uid 500); 3 Sep 2010 15:03:18 -0000
Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org
Received: (qmail 21401 invoked by uid 500); 3 Sep 2010 15:03:15 -0000
Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:solr-user-help@lucene.apache.org>
List-Unsubscribe: <mailto:solr-user-unsubscribe@lucene.apache.org>
List-Post: <mailto:solr-user@lucene.apache.org>
List-Id: <solr-user.lucene.apache.org>
Reply-To: solr-user@lucene.apache.org
Delivered-To: mailing list solr-user@lucene.apache.org
Received: (qmail 21393 invoked by uid 99); 3 Sep 2010 15:03:14 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Sep 2010 15:03:14 +0000
X-ASF-Spam-Status: No, hits=-0.0 required=10.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: local policy)
Received: from [67.23.2.84] (HELO mail.staffing-systems.co.uk) (67.23.2.84)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Sep 2010 15:03:05 +0000
Received: from [192.168.0.198] (80-42-243-226.dynamic.dsl.as9105.com
 [80.42.243.226])
	by mail.staffing-systems.co.uk (Postfix) with ESMTPSA id C2B7D111061
	for <solr-user@lucene.apache.org>; Fri,  3 Sep 2010 15:02:43 +0000 (UTC)
Message-ID: <4C810E11.7010604@staffing-systems.co.uk>
Date: Fri, 03 Sep 2010 16:02:41 +0100
From: Luke Tebbs <luke@staffing-systems.co.uk>
User-Agent: Mozilla-Thunderbird 2.0.0.24 (X11/20100328)
MIME-Version: 1.0
To: solr-user@lucene.apache.org
Subject: Re: Localsolr with Dismax => workaround using spatial solr
References: <4C7F8767.4050601@staffing-systems.co.uk>
 <4C7FBACB.9030608@adicio.com> <4C7FCF23.5080806@staffing-systems.co.uk>
In-Reply-To: <4C7FCF23.5080806@staffing-systems.co.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I finally managed to get spatial searching working in combination with 
dismax so I'm sending this should anyone else have the same problem.

I gave up using localsolr in the end - one of the resultsets of the two 
it returned was correct (dismax+spatial) but I don't trust this enough 
to depend upon it, don't want the surplus data and want to be able to 
sort by distance.

In the end I switched to using the spatial solr module. On my setup the 
default search is a dismax and the basedOn parameter for the spatial 
query parser is set to dismax. If you search without the spatial 
criteria it gives a dismax search as expected however when I try to use 
a spatial criteria it reverts to using a normal search despite the 
settings - much the same as with localsolr.

I seem to however have a working solution, albeit a not particularly 
pretty one, using filter queries -

<lst name="params">
    <str name="q">term</str>
    <str name="fq">{!spatial lat=50 long=-3 radius=100}*:*</str>
</lst>

However this does not allow you to sort against distance - presumably 
the filter is applied after the sort, instead I managed to get this 
working by swapping the two around and explicitly telling the filter to 
use dismax.

<lst name="params">
    <str name="sort">distance asc</str>
    <str name="q">{!spatial lat=50 long=-3 radius=100}*:*</str>
    <str name="fq">{!dismax}term</str>
</lst>

I don't know if this is suboptimal as the spatial search will likely be 
more expensive than the dismax (i think) but using the 20,000 odd 
records I'm testing with this is still ninja-quick. I'm going to up the 
dataset to a couple of million records and see if it is still acceptably 
fast.

Anyway, does anyone know if there is something I could be doing wrong 
that is causing dismax to not play nice with the two spatial searching 
methods, or is this one for the JIRA?

Luke



Luke Tebbs wrote:
> Thanks Dan,
> That seems to have moved things forwards, however if I do this I get 
> two <result> sets, presumably one from localsolr and one from dismax.
>
> e.g -
> <response>
> <responseHeader>
> <status>0</status>
> <QTime>116</QTime>
> </responseHeader>
> <result name="response" numFound="358" start="0">
> ...
> </result>
> <result name="response" numFound="8207" start="0">
> ...
> </result>
> </response>
>
> Also it seems to explode with a NullPointerException if I dare to try 
> and sort by distance -
> INFO: [testCore] webapp=/solr path=/select 
> params={sort=geo_distance+asc&q=some+phrase&radius=30&long=-0.1262362&qt=geo&wt=javabin&lat=51.5001524&rows=0&version=1} 
> status=500 QTime=123
> 02-Sep-2010 16:44:50 org.apache.solr.common.SolrException log
> SEVERE: java.lang.NullPointerException
>    at 
> org.apache.lucene.spatial.tier.DistanceFieldComparatorSource$DistanceScoreDocLookupComparator.copy(DistanceFieldComparatorSource.java:105) 
>
>    at 
> org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.collect(TopFieldCollector.java:84) 
>
>    at 
> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:292)
>    ....
>
> I don't know if these are related - perhaps it's trying to compare 
> against the dismax records that don't have a geo_distance?
> Did you get anything like this?
>
> Luke
>
> dan whelan wrote:
>>  I experienced the same issue. The localsolr site says to configure 
>> like this:
>>
>> <arr name="components">
>> <str>localsolr</str>
>> <str>facet</str>
>> <str>mlt</str>
>> <str>highlight</str>
>> <str>debug</str>
>> </arr>
>>
>> but the default solr components are (note the above config is missing 
>> query):
>>
>> query
>> facet
>> mlt
>> highlight
>> stats
>> debug
>>
>> I fixed it by doing this instead
>>
>> <arr name="first-components">
>> <str>localsolr</str>
>> </arr>
>>
>>
>>
>>
>>
>> On 9/2/10 4:15 AM, Luke Tebbs wrote:
>>> Anyone?
>>>
>>> I'm really lost as to what to do here... if anyone has any 
>>> experience with this
>>> or even ideas of things to try I'd really appreciate your input.
>>>
>>> It seems like what I'm trying to do should work but for some reason 
>>> 'defType' seems to be
>>> ignored....
>>>
>>> Thankyou
>>>
>>> Luke
>>>
>>> -------- Original Message -------
>>>
>>> Does anyone have any experience with getting dismax to work with a 
>>> geospatial (localsolr) search?
>>>
>>> I have the following configuration -
>>>
>>> ....
>>> <requestHandler name="standard" class="solr.SearchHandler" 
>>> default="true">
>>> <lst name="defaults">
>>> <str name="defType">dismax</str>
>>> <str name="qf">title description^0.5</str>
>>> <str name="pf">title description^0.5</str>
>>> <str name="mm">0%</str>
>>> <str name="tie">0.1</str>
>>> </lst>
>>> </requestHandler>
>>>
>>> <requestHandler name="geo" class="solr.SearchHandler">
>>> <lst name="defaults">
>>> <str name="defType">dismax</str>
>>> <str name="qf">title description^0.5</str>
>>> <str name="pf">title description^0.5</str>
>>> <str name="mm">0%</str>
>>> <str name="tie">0.1</str>
>>> </lst>
>>> <arr name="components">
>>> <str>localsolr</str> <str>facet</str>
>>> <str>mlt</str>
>>> <str>highlight</str>
>>> <str>debug</str>
>>> </arr>
>>> </requestHandler>
>>> ....
>>>
>>> All of the location searching works fine, as does the normal search, 
>>> but when using the "geo" handler the textual search seems to be 
>>> using the standard search handler and only the title field is searched.
>>>
>>> I'm a bit stumped on this one, any help would be greatly appreciated.
>>>
>>> Luke
>>>
>>
>