Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of siriele2x3@gmail.com
 designates 209.85.212.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJfs-5DP_nFJEVSBaJgkxkm3ex=0hgRYakL81iP05=60S9Hi6g@mail.gmail.com>
References: 
 <CAJfs-5AJ-MQdKNY8xu6DdA8kqikAb-47ZHuvg7m9znGAGo1big@mail.gmail.com>
	<BD1899F1-586F-499F-842E-37A311C4A9C5@programmazione.it>
	<CAPNuaChGau5ZVLUPxkgi5FN-j7yAbo8tYf3BL7d60XqeRs5+9g@mail.gmail.com>
	<CAJfs-5DP_nFJEVSBaJgkxkm3ex=0hgRYakL81iP05=60S9Hi6g@mail.gmail.com>
Date: Wed, 14 Aug 2013 21:00:19 -0700
Message-ID: 
 <CAPNuaCjR6Z4ZvwqnRvdZOegG5GGuHRESgwBOa8Se+Z7sB6Hdrg@mail.gmail.com>
Subject: Re: "reverse" view - find out views that a doc affects?
From: Stanley Iriele <siriele2x3@gmail.com>
To: user@couchdb.apache.org
Content-Type: multipart/alternative; boundary=047d7bfced1e32497604e3f48371

--047d7bfced1e32497604e3f48371
Content-Type: text/plain; charset=ISO-8859-1

Hey Dan,

there are a few steps you can take to help make that I tad faster or at
least I did something about this.
1) setup a crontab task to hit your design docs with HEAD call to make sure
the tree is always "just about" built.
2) when a View is returned it has a E-Tag and you can send that value with
a "If-None-Match" Header to see if the view has changed since you last
called it. It will return 304 with no content body sent.
3) use a list function to trim the fat off anything you don't need. also it
seems strange that it would take 3000ms to send something across the
wire...either its a lot of data or its taking along time to establish a
connection. you could try compressing it as well.

hope that helps.

Regards,
Stanley


On Wed, Aug 14, 2013 at 8:30 PM, Daniel Myung <dmyung@dimagi.com> wrote:

> Our issues with exploring caching "after" couch db stems from a variety of
> reasons.
>
> 1:  We do a lot of varied view calls on a single page to assemble our
> pages. We ajax the big stuff, but basic other parts that need to be timely
> and updated can routinely run between 80-250ms per view call.
> 2:  A view with an include_docs=True or even a single doc get runs 50-75 ms
> on a good day, and could be more if background reindexing or other burdens
> on our server are high.
> 3:  A lot of said views are already very efficient as you said since the
> couch view is already an index, but there is still a retrieval burden we
> experience. It's consistent and predictable, but it adds up.
> 4:  Our couch db server is running off an encrypted volume (ecryptfs - due
> to it hosting medical data) so we theorize that most of the OS level
> caching on views and docs for some reason we are not benefiting from at
> all.
> 5:  The contents of the frequently accessed documents and views we want are
> infrequently updated. Where queries are unavoidable like reduce views and
> frequently updated information (like encounters and such), we rely on the
> views directly and try to background load the UI where needed.
>
> Going back to point 4, our requests for pure couch retrieval can range from
> 800-3000 ms retrieving this information. A very simple in-memory caching
> test effectively eliminated that latency (don't have real numbers, but it
> is on the order of a 10-15x speed improvement for us.
>
> Dan
>
>
> On Wed, Aug 14, 2013 at 12:49 PM, Stanley Iriele <siriele2x3@gmail.com
> >wrote:
>
> > Is the bottleneck actual http round trips? Or view building?... If there
> > are no changes to said database... And the vies has been built it will be
> > fast....couchdb does a very good job about this... I'm not sure if it
> does
> > etaging for views but there is some equivalent.
> >
> > What is the problem your facing?
> > On Aug 14, 2013 9:31 AM, "Filippo Fadda" <
> filippo.fadda@programmazione.it>
> > wrote:
> >
> > > No, there is not. This is an information that knows only who wrote the
> > > view map function. I remember you, every document pass through each
> > views,
> > > and only there, inside the map function, you decide to emit or discard
> > the
> > > document from the view.
> > > If a view has been updated, even if the document you have inserted is
> not
> > > emitted for that view, this could be a bug and you should report it.
> The
> > > view is an index, and an index shouldn't change if you are not
> inserting
> > a
> > > new node.
> > >
> > > -Filippo
> > >
> > > On Aug 14, 2013, at 6:12 PM, Daniel Myung wrote:
> > >
> > > > Is there a way or a mechanism or trick or hack of some sort to find
> out
> > > for
> > > > a given doc I deposit to couch to find out what views it affects? Is
> > > there
> > > > a record of doc ids being operated upon in a design doc reindexing
> > > > operation?
> > > >
> > > > for various reasons, we're looking to reduce the number of view call
> > > round
> > > > trips that our application does on couch, and were looking to cache
> the
> > > > view response...but invalidation due to *new* docs that could show in
> > the
> > > > view (that we haven't seen) has been preventing us from going further
> > > down
> > > > this route.
> > > >
> > > > Thanks
> > > >
> > > > Dan
> > >
> > >
> >
>

--047d7bfced1e32497604e3f48371--