lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Male <gento...@gmail.com>
Subject Re: [spatial] Rethinking Cartesian Tiers implementation
Date Wed, 30 Jun 2010 07:51:47 GMT
Hi Grant,

I'm with you in agreement on this issue.  The tier implementation has been
functional while people have remained within quadrants, which just happens
to be the case with the US, Europe, and large parts of Asia.

My concern is not so much that its broken when crossing quadrants, but that
we don't have the test coverage at each step in the process to verify that
its working correctly.  To get the test coverage we need knowledge about how
its supposed to work.  Although we can work on getting that, we don't have
it today and I don't want to see Solr's support held up any longer.

How much of the spatial contrib are you using in Solr? Just the geohashing
functionality?  I wonder because given that I have a whole stack of
improvements for spatial pending, which basically form a total re-write of
the codebase.

Consequently is it not a better idea to deprecate the entire contrib, take
what parts we know are functional and tested, and create a first-class
spatial module.  We can then explore other options such as the morton number
filtering I have, prefix queries for geohashes, and all other exotic
bounding box hacks I've heard about from Uwe and Robert.  And while we
explore these options, we can make sure they are thoroughly tested, and not
overly complex before we add them to the module slowly.

This way people who are using the contrib successfully can continue too, and
then we can maybe drop the contrib from the 4.x releases.

Cheers
Chris

On Wed, Jun 30, 2010 at 5:02 AM, Grant Ingersoll <gsingers@apache.org>wrote:

> After having spent a lot of time looking at the Tier stuff and debugging it
> and discussing it on IRC, I am of the opinion (and, I still have some belief
> that it's just me who isn't getting it, but others have confirmed my
> suspicion) that the spatial Cartesian Tier implementation is broken and, due
> to the lack of documentation and resources, it is not clear how to fix it;
> furthermore we don't seem to have anyone with the bandwidth or interest to
> fix it (I've tried, but I've about reached my limit on it in terms of
> willingness to work on it).  Even if it is not broken and it is just me, the
> fact that I don't get it after spending as much time on it that I have is
> not a good sign for maintainability.
>
> AFAICT, a lot of it stems from the fact that SinusoidalProjection is broken
> (https://issues.apache.org/jira/browse/LUCENE-2519) and then layers upon
> layers of other, minor issues were built on top of that such that at the end
> of those layers are unit tests that are nearly impossible to untangle and in
> between is a fairly large mess.   Well, I suppose I should moderate those
> statements: I think all of it probably works as long as you stay completely
> in a given quadrant of the Earth for all your points (and likely really,
> just the US/ northern/western hemispheres), but even then it only works b/c
> it consistently calculates the wrong values for all points.
>
> At the same time, I think the theory behind it (i.e. a labeled, nested grid
> system) is useful for many people solving spatial search problems and can
> offer significant query time savings.  So, I think there are a couple of
> things we could do:
>
> 1. Remove it and start from scratch and take a more standard approach
> 2. Find someone to fix the existing stuff who groks it
> 3. Document that it is only really useful for a given quadrant and then
> either do 1 or 2 over the long term.  We still need to fix somethings, IMO,
> to feel comfortable w/ it.
> 4. At a minimum, I think we should mark it as deprecated until it can be
> better dealt with later
>
> For now, I'm going to remove it from Solr and just focus on finishing up
> the filtering stuff for the other point types.
>
> Thoughts?  Volunteers?
>
> -Grant
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>


-- 
Chris Male | Software Developer | JTeam BV.| www.jteam.nl

Mime
View raw message