couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristopher Tate <kristop...@bluebridge.jp>
Subject Re: Updating views on save
Date Mon, 28 Apr 2008 03:19:26 GMT
Sebastian, at the moment I am following developments just like you and  
starting to study more of the codebase.
This may or may not be correct, but I believe that the benefits of  
having views update on the first time they are viewed far outweighs  
updating the index/view each time data gets stored into CouchDB.

In the current situation, if I write 10 times and read 100 times, the  
index may only be processed once if the 100 reads come after the 10  
writes -- in a more real-world situation though, we're only going to  
be updating the index as many times as we write -- but keep in mind  
that given the right circumstances, the index only has to be  
regenerated once.

If the DB indexed for each write, the number of indexes is the same as  
writes, which may incur more IO penalties, beyond a simple write  
(EmitReduce, etc.)

Also, please realize that the index does not have to be /completely/  
regenerated upon reindex -- only the documents that have been added/ 
modified.

Optional updating (as you mention) could be a compromise, but I think  
that resources should be spent making reindexing views faster.

Lastly, I would recommend that you try to optimize your view code to  
make things faster, as well.

kristopher

On 2008/04/28, at 8:04, Guby wrote:
> Hello dear Couchers
>
> I understand that the views are indexed the first time they are  
> accessed and as far as I know there is no way to turn on view  
> updating on document save. I really don't understand the reasoning  
> behind this behavior. The advantage of the pre-populated/indexed  
> views are that they are blazingly fast to query and access, but that  
> advantage disappears when the first request after a document update  
> has to regenerate the view first!
>
> I am currently building a web app where the background processes  
> perform a lot of writes to the database. The time it takes to write  
> a document is not critical for me. What is critical though is the  
> time it takes to load web pages for the end user that require  
> content from the database.
> In some situations the background processes add thousands of  
> documents to the database within a short period of time, and when  
> the user tries to access a page after such an update the view  
> querying sometimes takes minutes and as a consequence of that the  
> browser times out... Not a recipe for happy customers...
>
> The only solution I can see at the moment is to create a worker that  
> queries the database whenever it is told that there has been a  
> document update, but that seems really stupid and unnecessary. And  
> in my case, running on a smallish VPS, a big waste of resources  
> having an extra working doing something the database itself could  
> just as well have done. It also requires a lot of extra coding  
> notifying the worker whenever I update or create a document  
> throughout my app.
>
> I am sure you have reasons for having implemented the views the way  
> you have, but I would be really interested to hear why it has been  
> done this way!
>
> My wishes are for an optional updating of views on save feature! In  
> some cases that might regenerate a view several times without it  
> actually being accessed in between, but that is a tradeoff I can  
> live with, slow views on the other hand is something I can not!
>
> All the best
> Sebastian


Mime
View raw message