couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mitja Kleider <mi...@kleider.name>
Subject Re: Replication filter ignoring deleted docs or changed values
Date Wed, 13 Jul 2011 13:52:09 GMT
Zachary,

On Sun, 2011-07-10 at 13:35 -0500, Zachary Zolton wrote:
> Mitja,
> 
> Perhaps you could record in some other field that the doc with type
> "bar" used to be type "foo".
> 
> Then way you can filter the replication by docs where:
>   type == "foo" || type == "bar" && old_type == "foo"
> 
> This should then replicate docs that change their type from "foo" to "bar".
> 
> To give any better advice, I'd need to hear more concretely what
> you're trying to do here.

I am working with geospatial data from the OpenStreetMap project.

Each document has infinitely many tags, similar to type="restaurant"
which might change in each revision.

I would also like to filter the geometry, to keep it simple something
like [x,y] and return (left <= x) && (x <= right) &&  (bottom <=
y) &&
(y <= top).

The position might also change. I am afraid that I can not keep the
entire history because of the high amount of data.

The only possible other solution I have come up with is not using
filtered replication for updating the replicated target database, but
instead do full replication and filter in validate_doc_update(). Does
that make sense?


Mitja


> On Sunday, July 10, 2011, Mitja Kleider <mitja@kleider.name> wrote:
> > On Sat, 2011-07-09 at 17:26 -0500, Zachary Zolton wrote:
> >> Could you change your filter to something like this?
> >>
> >> function(doc) {
> >>   return doc._deleted || doc.type == 'foo';
> >> }
> >>
> >> That way you replicate all deleted docs.
> >
> > That would work, thanks. I tried to avoid replicating millions of
> > deleted documents together with a few hundred documents passing the
> > filter.
> >
> > A solution might be this validate_doc_update function on the target
> > database:
> >
> > function(newDoc, oldDoc, userCtx) {
> >     if (newDoc._deleted === true && !oldDoc) {
> >         throw({forbidden: 'do not create deleted docs'});
> >     }
> > }
> >
> > This way all deleted documents will be transmitted, but at least not
> > written if there is no previous revision.
> >
> >
> > The problem for changed values remains: when the source database changes
> > type='foo' to type='bar', the target database keeps the old revision
> > with type='foo'.
> >
> >
> > Mitja
> >
> >




Mime
View raw message