couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject RIP Temp Views?
Date Wed, 15 Jul 2009 15:57:44 GMT
Hi Couchers,

we're getting into a phase where CouchDB is distributed in software
distributions (like Ubuntu) with long term support on the distributors
end. While this is in theory none of our concern, we probably want
to make sure our future is as smooth as theirs (i.e. users are likely
to come to this community for help and with problems, even though
we only got into their hands by proxy.

One of the things I want to bring up here. Features of CouchDB that
we're not comfortable supporting in the long term. While we can leave
that for "before 1.0", I think timing requires being a little more  
upfront.

(One of the examples of a feature we were not comfortable supporting
was the transactional bulk docs. While the jury is still out, the  
feature
is only available via Antony's patch for the time being).

A feature I am not comfortable with is temp views. The approach
renaming them with slow views didn't pan out so I'm proposing to get
rid of them all together.

I appreciate the usefulness, but I think we can achieve the same
flexibility and playaroundability without temp views and even gain
something.

One the plus side, newcomers repeatedly run into temp view
performance problems. In earlier discussions I proposed that while
documentation helps making clear users shouldn't use them, we can't
control how users learn CouchDB, but we do control the API. By not
giving them rope to hang, we save users. Besides, nobody reads the
bloody docs.

The last months (I spent ~12 weeks on the road talking about
CouchDB to everybody who wanted to listen this year) showed me
that I was right (yay me :-).

Just as an example, from today's #couchdb IRC channel:

   …
   tahorg__ joined the chat room.
   …
   tahorg__: Hi, I'm trying couchdb and I'm having some _heavy_  
performances issue
   jan____: don't use temp views
   …

Down the road, it was temp views. q.e.d.

--

RIP Temp Views. The core problem with temp view is them running
a full "table scan". This works for 100 and 1000 docs, but gets
uncomfortable for 5000 and more.

Temp views are also great for trying things out and Futon makes it
super simple to get going. We should keep the easy without the pain.

Here is how we can keep temp view behaviour without temp views:
Take a JavaScript implementation and put it into jquery,couch.js
that acts as a replication endpoint for a real CouchDB and calculates
results based on the user-provided map- and reduce-functions.

A replication-pull endpoint is just a loop over the _all_docs_by_seq
view. I propose the implementation should default to only replicate
the last 100 or 1000 docs from a database into memory and run the
view computation there.

Luckily there is a CouchDB view engine written in JS already* (live
demo **) that we can just integrate. I started the work on GitHub*.
I started replacing the view(map_fun..) function with the JS version
explained above.

* http://github.com/janl/jscouch/tree
** http://labs.mudynamics.com/2009/04/03/interactive-couchdb/
*** http://github.com/janl/couchdb/tree/rip-temp-views

The reason I started in couch.js (as opposed to jquery.couch.js) is
me wanting to get the implementation right by getting the test suite
to pass. Porting to the jQuery plugin should be trivial once done.

Futon doesn't even have to change for that.

--

But what if we just change temp view to only look at the 1000
latest docs? It still gives users the wrong idea about querying
CouchDB and will get a major WTF? once they go beyond
the temp view doc limit.

--


Is this is a cause you'd like to see finished and are you a JS-head
that likes to help out getting the rest of the test suite to pass  
(basically
only grouped reduce hasn't been ported from jscouch yet) please
speak up so we can organize the effort.

If you like temp views, I'd like to take your concerns into  
consideration.


RIP Temp View?

Cheers
Jan
--








Mime
View raw message