lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin Freas <>
Subject Re: Using Solr's spatial functionality for astronomical catalog
Date Tue, 16 Feb 2016 08:31:47 GMT

David, thanks for getting back to me.  SpatialRecursivePrefixTreeFieldType
seems to be what I need, and the default search seems appropriate.  This
is for entries in an astronomical catalog, so great circle distances on a
perfect sphere is what I¹m after.

I am having a bit of difficulty though.

Having gotten records importing via our database into a schema on both a
stand-along Solr instance and in a SolrCloud cluster, I¹ve moved on to
³spatializing² the appropriate fields, and everything looks like it¹s
working, in that there are no errors thrown.  But when I try what I think
is valid spatial query, it doesn¹t work.

Here¹s what I¹m doing.  Pertinent bits from my schema:

		<field name="SourceRpt" type="rpt" indexed="true" stored="false"
required="false" multiValued="false" />
		<fieldType name="rpt" class="solr.SpatialRecursivePrefixTreeFieldType"
			distanceUnits="degrees" />

In my db-config.xml, I¹ve got this sql:
	<entity name="observation" query="SELECT CatID, MatchRA, MatchDec,
SourceRA, SourceDec,  '''' + ltrim(rtrim(Str(SourceRA,25,16))) + ',' +
ltrim(rtrim(Str(SourceDec,25,16))) + '''' FROM dbo.Catalog" />

When I run a data import through Solr¹s admin gui and look at the verbose
debug output, something seems off.  Top of the output is this:
	"responseHeader": {
	"status": 0,
	"QTime": 75
	"initArgs": [
	"command": "full-import",
	"mode": "debug",
	"documents": [
	"MatchDec": [
	"SourceDec": [
	"MatchRA": [
	"SourceRA": [
	"CatID": [

There¹s no SourceRpt field there.  But in verbose output of what¹s
returned from the query, the SourceRpt field seems to be correctly put

	"verbose-output": [
	"SELECT CatID, MatchRA, MatchDec, SourceRA, SourceDec,  '''' +
ltrim(rtrim(Str(SourceRA,25,16))) + ',' +
ltrim(rtrim(Str(SourceDec,25,16))) + '''' FROM xcat.BestCatalog",
	"----------- row #1-------------",

I try a spatial search like this:

... And I get back all (10) records in the core, when I would expect 0,
given the very small distance I supply to a point well away from any of
the records.

I¹m not sure what¹s going on.  I don¹t know if this is a simple Solr
config error I¹m missing, or if there¹s some spatial magic I¹m unaware of.

Any thoughts appreciated.


On 1/20/16, 9:34 PM, "" <>

>Hello Colin,
>If the spatial field you use is the SpatialRecursivePrefixTreeFieldType
>(RPT for short) with geo="true" then the circle shape (i.e. point-radius
>filter) implied by the geofilt Solr QParser is on a sphere.  That is, it
>uses the "great circle" distance computed using the Haversine formula by
>default, though it can be configured to use the Law of Cosines formula or
>Vincenty (spherical version) formula if you so choose.  Using geodist()
>spatial distance sorting/boosting also uses this.  If you use LatLonType
>then geofilt & geodist() use Haversine too.
>If you use polygons or line strings, then it's *not* using a spherical
>model; it's using a Euclidean (flat) model on plate carrée.  I am
>working on adapting the Spatial4j library to work with Lucene's Geo3D (aka
>spatial 3d) which has both a spherical model and an ellipsoidal model,
>which can be configured with the characteristics specified by WGS84.  If
>you are super-eager to get this yourself without waiting, then you could
>write a Solr QParser that constructs a Geo3dShape wrapping a Geo3D
>object constructed from query parameters.  You might alternatively try and
>use Geo3DPointField on Lucene 6 trunk.
>~ David
>On Tue, Jan 19, 2016 at 11:07 AM Colin Freas <> wrote:
>> Greetings!
>> I have recently stood up an instance of Solr, indexing a catalog of
>> 100M records representing points on the celestial sphere.  All of the
>> fields are strings, floats, and non-spatial types.  I¹d like to convert
>> positional data to an appropriate spatial point data type supported by
>> I have a couple of questions about indexing spatial data using Solr,
>> it seems spatial4j, and the spatial functionality in Solr generally, is
>> more GIS geared.  I worry that the measurements of lat/long on the
>> imperfect sphere of the Earth wouldn¹t match up with the astronomical
>> ascension/declination concept of the perfectly spherical celestial
>> used to record the coordinates of our records.
>> I¹m also worried there might be other assumptions built into spatial4j &
>> Solr based on using a real surface vs a virtual one.
>> Does anyone have experience doing this, or is there perhaps some
>> documentation specific to this use case that anyone might be able to
>> me to?
>> Thanks in advance,
>> Colin
>Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>LinkedIn: | Book:

View raw message