couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pablo platt <pablo.pl...@gmail.com>
Subject Re: using couchdb for 100GB geodata from openstreetmap
Date Wed, 25 Mar 2009 15:22:42 GMT
On Tue, Mar 24, 2009 at 11:08 PM, Wout Mertens <wout.mertens@gmail.com>wrote:

> How about this:
>
> 1. Create view functions that map coordinates in square cells that contain
> "not too many" results, create an X,Y and Y,X index
>
> 2. Create reduce functions that count number of items in those cells
>
> 3. Now query for everything in a rectangle or square by asking an _external
> process
>
> 4. The _external process does local queries.
> 4a. First it finds out if there's less results when querying on X or on Y
> by getting the count
> 4b. Then for the selected range (let's say X), query
> [startX+i,startY...endY] where i goes from 0 to endX-startX
> 4c. Filters out the types of documents that are requested and returns them.
>
> If there's not too many types you can make X,Y views per type.
>
> Is that naive? As long as the cells are not too large and not too small,
> you wouldn't be processing much more documents than you would actually need
> to process and using an _external process keeps the back-and-forth view
> traffic very local.
>
> Wout.
>

This is how openstreetmap 'Name Finder' works currently but it's not as
flexible as a spatial aware database.
PostGIS/PostgreSQL which is the best open source spatial aware db I know of
works with R-Tree-over-GiST scheme which should be comparable with R-tree.
PostGIS let you perform complicated spatial queries.

Does anyone have an idea how Hilbert curves and morton numbers compare to
R-tree and if they have any limitations?
Will using morton numbers remove the need of an external view engine?



>
> On Mar 23, 2009, at 3:20 PM, Volker Mische wrote:
>
>  James Marca wrote:
>>
>>> On Mon, Mar 23, 2009 at 12:07:46AM +0100, Volker Mische wrote:
>>>
>>>> Is there a difference between a custom view engine and the approach I
>>>> try to pursue with GeoCouch? It still comes down to the fact that you
>>>> need to store the spatial information in some kind of (external) spatial
>>>> index.
>>>>
>>>
>>>  [...]
>>
>>>
>>> The Geo Couch approach, as I understand it (again, forgive any
>>> ignorance on my part) is an external process that responds to queries,
>>> right?  So there isn't an index that CouchDB understands, and you
>>> can't run map/reduce type queries over it.  At least, that's why I
>>> didn't try to load up GeoCouch.  I was looking for a view engine that
>>> was a bit more native to the map/reduce aspects of CouchDB.
>>>
>>
>> This is how it works at the moment. The next thing I'll be working on is
>> _external and view intersection. This means that the external process
>> returns some results which will be combined with the results of a view.
>> This way you get (hopefully) the power of CouchDB for attribute queries,
>> but high speed for the spatial part of the query.
>>
>> Cheers,
>>  Volker
>>
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message