Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of
 gcdcu-couchdb-user@m.gmane.org designates 80.91.229.12 as permitted sender)
To: user@couchdb.apache.org
From: Nicolas Jessus <nicolas.jessus@lores.org>
Subject: Re: Forcing document reindex
Date: Wed, 17 Nov 2010 16:13:47 +0000 (UTC)
Lines: 31
Message-ID: <loom.20101117T162904-685@post.gmane.org>
References: <loom.20101117T134738-719@post.gmane.org>
 <76A109FD-9829-4EAA-9BA1-0FAC29357EA9@apache.org>
 <loom.20101117T142932-408@post.gmane.org>
 <7D7C2F35-4630-494D-BD39-C446FCB3486E@apache.org>
 <loom.20101117T151423-281@post.gmane.org>
 <D7D21638-2CC6-4F68-B318-F3A8FC000B12@apache.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
User-Agent: Loom/3.14 (http://gmane.org/)

All right; no one should like what they're going to read.

I have a medium-sized MySQL system, which translates to a Couch with about a
million documents of about 20 types. The system would really benefit from a
schema-free design. The data is only weakly relational. Couch would fit really
well, enough that I don't mind twisting its arm in a few places if need be; the
tradeoff would be worth it. 

The hiccup is reporting. Some of it involves the full set of documents. Let's
say I have 5 categories of documents involved in a report, A to E. A links to B,
B links to C, etc. The report needs data from A, B, and E. As far as I can
think, there's no way to do a view collation, because A and B share an ID but E
doesn't. I can't pull a million documents from the DB to process elsewhere
either, so that nixes simple indexing and the '_id' object values. 

I could however write a special view_server that will emit keys after checking
the linked ID through an HTTP call (that's where you scream). Indexing
performance is totally unimportant to me, DB updates are relatively few, and I
can live with the dirty side-effects (again, the system as a whole would still
be much cleaner than the MySQL one). 

With that solution I can have a map function that just handle docs of type A.
But I still need to reindex the relevant As when B or E changes. I could simply
listen to the change stream and force a reindex, but that doesn't work well with
legitimate updates when the _rev number goes up at random even though the doc
hasn't changed, and there's no auto-merge. So I'm pretty stuck.

I'm not asking that this type of functionality be encouraged. It's clearly
subverting the point of Couch. On the other hand, it doesn't seem like having a
force-reindex function would dirty the concept, and if it's easy to code, then
it's a shame it doesn't exist.