couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pablo platt <pablo.pl...@gmail.com>
Subject using couchdb for 100GB geodata from openstreetmap
Date Sun, 22 Mar 2009 21:39:51 GMT
Hi,

I'm trying to understand if couchdb is suitable for storing
openstreetmap(OSM) xml data.
OSM xml file contains geodata of the whole world. it's size is over 100GB
and increasing each year.

For simplicity it has 3 types of elements(
http://wiki.openstreetmap.org/wiki/Data_Primitives):
1. Node - defines a point(lat, lon) on the map and can but don't have to
contain tags.
A node without tags - used as points in a way for example:
<node id="139698" timestamp="2009-03-11T18:07:59Z" user="talkaat"
lat="15.0938835" lon="15.7912674"/>
A node with tags - used to describe a place:
<node id="139698" timestamp="2009-03-11T18:07:59Z" user="talkaat"
lat="15.0938835" lon="15.7912674"/>
<tag k="name:de" v="Paris"/>
<tag k="name:fr" v="Paris"/>
<tag k="name:en" v="Paris"/>
</node>

2. Way - a collection of nodes describing a road:
<way id="5090250" visible="true" timestamp="2008-05-03T12:16:45+01:00"
user="Andy Allan">
<nd ref="822403"/>
<nd ref="21533912"/>
...
<tag k="oneway" v="yes"/>
<tag k="highway" v="unclassified"/>
<tag k="created_by" v="Potlatch 0.8c"/>
<tag k="name" v="Clipstone Street"/>
</way>

3. Relation - define  relationship between objects:
<relation id="12176" visible="true" timestamp="2008-08-09T17:55:39+01:00"
user="Andy Allan">
<member type="way" ref="4068452" role="forward"/>
<member type="way" ref="4253466" role=""/>
...
 <tag k="ref" v="0"/> <tag k="type" v="route"/>
<tag k="created_by" v="Potlatch 0.10b"/>
 <tag k="route" v="bicycle"/>
</relation>

The OSM XML is pretty compact. The timestamp and user attributes can be
removed and it could decrease the db size by ~1/2
Can couchdb host 100GB of data?
Is the correct way to do store the data is to use a separate doc for each
element(point, way, relation)?
If the db will be used only for reading and replication is not a
requirement, does it changes anything?

I've read about geocouch -
http://vmx.cx/cgi-bin/blog/index.cgi/geocouch-geospatial-queries-with-couchdb:2008-10-26:en,CouchDB,Python,geo
and my question is if a geo aware couchdb can be built without any python
and SpatiaLite code?

If I want to be able to make proximity searches - find banks in 3km range
from st some-street
do I need to create some kind of index docs for the views?

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message