couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: RIP Temp Views?
Date Wed, 15 Jul 2009 16:43:51 GMT
Hi Paul,

thanks :)

On 15 Jul 2009, at 18:33, Paul Davis wrote:

> I like the idea of removing temp view code a lot.

I remember you saying that it'd remove some significant amount
of code out of the view server. This is something that should be
considered, too.


> The last time I
> remember this conversation it was basically limited by the fact that
> we wanted to keep supporting temp_views so that new users could use
> them to learn CouchDB. What we didn't have before were index files
> named after signatures. With a bit of work in Futon we could easily
> replace the current temp view interface with calls to the permanent
> view API.


> I'm thinking something along the lines of having the Futon interface
> create _design documents that include a "futon": true member so that
> we can filter them out in other parts of the interface, plus a list of
> "Old temp views" that you could pull up and edit. This would also give
> us the ability to have a nice button that says "Make this view
> permanent" or some such and then just have a delete button next to
> temp view names.
>
> Using a JavaScript API to replicate N documents to the browser to play
> with could be fun, but I'm hesitant that it'll end up causing more
> errors than are worth it when it feels like there's are easier ways to
> do the testing. If we wanted to do "replicate N docs" I would be more
> interested in a _decimate call that copied some number of documents to
> a new database that would allow people to play in a sandbox in that
> manner.

The bigger reason for that is that we don't have to rewrite half of the
test suite to support this. And to have a simplest-case example for
other third party libraries. While the replace-with-permanent-views
would work for Futon, it is probably not the best blueprint for other
libs. But I can be convinced otherwise.

Cheers
Jan
--


>
> Paul Davis
>
>
>
> On Wed, Jul 15, 2009 at 11:57 AM, Jan Lehnardt<jan@apache.org> wrote:
>> Hi Couchers,
>>
>> we're getting into a phase where CouchDB is distributed in software
>> distributions (like Ubuntu) with long term support on the  
>> distributors
>> end. While this is in theory none of our concern, we probably want
>> to make sure our future is as smooth as theirs (i.e. users are likely
>> to come to this community for help and with problems, even though
>> we only got into their hands by proxy.
>>
>> One of the things I want to bring up here. Features of CouchDB that
>> we're not comfortable supporting in the long term. While we can leave
>> that for "before 1.0", I think timing requires being a little more  
>> upfront.
>>
>> (One of the examples of a feature we were not comfortable supporting
>> was the transactional bulk docs. While the jury is still out, the  
>> feature
>> is only available via Antony's patch for the time being).
>>
>> A feature I am not comfortable with is temp views. The approach
>> renaming them with slow views didn't pan out so I'm proposing to get
>> rid of them all together.
>>
>> I appreciate the usefulness, but I think we can achieve the same
>> flexibility and playaroundability without temp views and even gain
>> something.
>>
>> One the plus side, newcomers repeatedly run into temp view
>> performance problems. In earlier discussions I proposed that while
>> documentation helps making clear users shouldn't use them, we can't
>> control how users learn CouchDB, but we do control the API. By not
>> giving them rope to hang, we save users. Besides, nobody reads the
>> bloody docs.
>>
>> The last months (I spent ~12 weeks on the road talking about
>> CouchDB to everybody who wanted to listen this year) showed me
>> that I was right (yay me :-).
>>
>> Just as an example, from today's #couchdb IRC channel:
>>
>>  …
>>  tahorg__ joined the chat room.
>>  …
>>  tahorg__: Hi, I'm trying couchdb and I'm having some _heavy_  
>> performances
>> issue
>>  jan____: don't use temp views
>>  …
>>
>> Down the road, it was temp views. q.e.d.
>>
>> --
>>
>> RIP Temp Views. The core problem with temp view is them running
>> a full "table scan". This works for 100 and 1000 docs, but gets
>> uncomfortable for 5000 and more.
>>
>> Temp views are also great for trying things out and Futon makes it
>> super simple to get going. We should keep the easy without the pain.
>>
>> Here is how we can keep temp view behaviour without temp views:
>> Take a JavaScript implementation and put it into jquery,couch.js
>> that acts as a replication endpoint for a real CouchDB and calculates
>> results based on the user-provided map- and reduce-functions.
>>
>> A replication-pull endpoint is just a loop over the _all_docs_by_seq
>> view. I propose the implementation should default to only replicate
>> the last 100 or 1000 docs from a database into memory and run the
>> view computation there.
>>
>> Luckily there is a CouchDB view engine written in JS already* (live
>> demo **) that we can just integrate. I started the work on GitHub*.
>> I started replacing the view(map_fun..) function with the JS version
>> explained above.
>>
>> * http://github.com/janl/jscouch/tree
>> ** http://labs.mudynamics.com/2009/04/03/interactive-couchdb/
>> *** http://github.com/janl/couchdb/tree/rip-temp-views
>>
>> The reason I started in couch.js (as opposed to jquery.couch.js) is
>> me wanting to get the implementation right by getting the test suite
>> to pass. Porting to the jQuery plugin should be trivial once done.
>>
>> Futon doesn't even have to change for that.
>>
>> --
>>
>> But what if we just change temp view to only look at the 1000
>> latest docs? It still gives users the wrong idea about querying
>> CouchDB and will get a major WTF? once they go beyond
>> the temp view doc limit.
>>
>> --
>>
>>
>> Is this is a cause you'd like to see finished and are you a JS-head
>> that likes to help out getting the rest of the test suite to pass  
>> (basically
>> only grouped reduce hasn't been ported from jscouch yet) please
>> speak up so we can organize the effort.
>>
>> If you like temp views, I'd like to take your concerns into  
>> consideration.
>>
>>
>> RIP Temp View?
>>
>> Cheers
>> Jan
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>


Mime
View raw message