couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: RIP Temp Views?
Date Wed, 15 Jul 2009 17:08:08 GMT
On Wed, Jul 15, 2009 at 12:43 PM, Jan Lehnardt<jan@apache.org> wrote:
> Hi Paul,
>
> thanks :)
>
> On 15 Jul 2009, at 18:33, Paul Davis wrote:
>
>> I like the idea of removing temp view code a lot.
>
> I remember you saying that it'd remove some significant amount
> of code out of the view server. This is something that should be
> considered, too.
>
>
>> The last time I
>> remember this conversation it was basically limited by the fact that
>> we wanted to keep supporting temp_views so that new users could use
>> them to learn CouchDB. What we didn't have before were index files
>> named after signatures. With a bit of work in Futon we could easily
>> replace the current temp view interface with calls to the permanent
>> view API.
>
>
>> I'm thinking something along the lines of having the Futon interface
>> create _design documents that include a "futon": true member so that
>> we can filter them out in other parts of the interface, plus a list of
>> "Old temp views" that you could pull up and edit. This would also give
>> us the ability to have a nice button that says "Make this view
>> permanent" or some such and then just have a delete button next to
>> temp view names.
>>
>> Using a JavaScript API to replicate N documents to the browser to play
>> with could be fun, but I'm hesitant that it'll end up causing more
>> errors than are worth it when it feels like there's are easier ways to
>> do the testing. If we wanted to do "replicate N docs" I would be more
>> interested in a _decimate call that copied some number of documents to
>> a new database that would allow people to play in a sandbox in that
>> manner.
>
> The bigger reason for that is that we don't have to rewrite half of the
> test suite to support this. And to have a simplest-case example for
> other third party libraries. While the replace-with-permanent-views
> would work for Futon, it is probably not the best blueprint for other
> libs. But I can be convinced otherwise.
>

Good call on considering the test suite's use of temp_views. Though
I'd have to look harder to see if there's a reason we can't just make
the db.query() do the same creation of a design document behind the
scenes.

For library support I would probably just refer people to the
_decimate type endpoint to create a sandbox. Though the big issue with
_decimate is that there probably isn't a widely usable scheme for
decimation which just makes me want to tell people to deal with it on
their own. Maybe we could add more general "temp": true (instead of
"futon": true) and supporting mechanisms, but I'm not overly
concerned. My main goal would be to make Futon's interface work first,
and then let other libraries copy that scheme.

Also, my original idea was to make the _temp_view end point create the
design documents so that the entire change would be transparent to all
current code except for the need to keep track of the current set of
_design documents. The previous criticisms of that were that it'd
introduce messiness in terms of handling spurious design documents,
though that argument might be rendered invalid by the fact that we now
have to do cleanup calls for old design doc indexes.

Paul Davis

> Cheers
> Jan
> --
>
>
>>
>> Paul Davis
>>
>>
>>
>> On Wed, Jul 15, 2009 at 11:57 AM, Jan Lehnardt<jan@apache.org> wrote:
>>>
>>> Hi Couchers,
>>>
>>> we're getting into a phase where CouchDB is distributed in software
>>> distributions (like Ubuntu) with long term support on the distributors
>>> end. While this is in theory none of our concern, we probably want
>>> to make sure our future is as smooth as theirs (i.e. users are likely
>>> to come to this community for help and with problems, even though
>>> we only got into their hands by proxy.
>>>
>>> One of the things I want to bring up here. Features of CouchDB that
>>> we're not comfortable supporting in the long term. While we can leave
>>> that for "before 1.0", I think timing requires being a little more
>>> upfront.
>>>
>>> (One of the examples of a feature we were not comfortable supporting
>>> was the transactional bulk docs. While the jury is still out, the feature
>>> is only available via Antony's patch for the time being).
>>>
>>> A feature I am not comfortable with is temp views. The approach
>>> renaming them with slow views didn't pan out so I'm proposing to get
>>> rid of them all together.
>>>
>>> I appreciate the usefulness, but I think we can achieve the same
>>> flexibility and playaroundability without temp views and even gain
>>> something.
>>>
>>> One the plus side, newcomers repeatedly run into temp view
>>> performance problems. In earlier discussions I proposed that while
>>> documentation helps making clear users shouldn't use them, we can't
>>> control how users learn CouchDB, but we do control the API. By not
>>> giving them rope to hang, we save users. Besides, nobody reads the
>>> bloody docs.
>>>
>>> The last months (I spent ~12 weeks on the road talking about
>>> CouchDB to everybody who wanted to listen this year) showed me
>>> that I was right (yay me :-).
>>>
>>> Just as an example, from today's #couchdb IRC channel:
>>>
>>>  …
>>>  tahorg__ joined the chat room.
>>>  …
>>>  tahorg__: Hi, I'm trying couchdb and I'm having some _heavy_
>>> performances
>>> issue
>>>  jan____: don't use temp views
>>>  …
>>>
>>> Down the road, it was temp views. q.e.d.
>>>
>>> --
>>>
>>> RIP Temp Views. The core problem with temp view is them running
>>> a full "table scan". This works for 100 and 1000 docs, but gets
>>> uncomfortable for 5000 and more.
>>>
>>> Temp views are also great for trying things out and Futon makes it
>>> super simple to get going. We should keep the easy without the pain.
>>>
>>> Here is how we can keep temp view behaviour without temp views:
>>> Take a JavaScript implementation and put it into jquery,couch.js
>>> that acts as a replication endpoint for a real CouchDB and calculates
>>> results based on the user-provided map- and reduce-functions.
>>>
>>> A replication-pull endpoint is just a loop over the _all_docs_by_seq
>>> view. I propose the implementation should default to only replicate
>>> the last 100 or 1000 docs from a database into memory and run the
>>> view computation there.
>>>
>>> Luckily there is a CouchDB view engine written in JS already* (live
>>> demo **) that we can just integrate. I started the work on GitHub*.
>>> I started replacing the view(map_fun..) function with the JS version
>>> explained above.
>>>
>>> * http://github.com/janl/jscouch/tree
>>> ** http://labs.mudynamics.com/2009/04/03/interactive-couchdb/
>>> *** http://github.com/janl/couchdb/tree/rip-temp-views
>>>
>>> The reason I started in couch.js (as opposed to jquery.couch.js) is
>>> me wanting to get the implementation right by getting the test suite
>>> to pass. Porting to the jQuery plugin should be trivial once done.
>>>
>>> Futon doesn't even have to change for that.
>>>
>>> --
>>>
>>> But what if we just change temp view to only look at the 1000
>>> latest docs? It still gives users the wrong idea about querying
>>> CouchDB and will get a major WTF? once they go beyond
>>> the temp view doc limit.
>>>
>>> --
>>>
>>>
>>> Is this is a cause you'd like to see finished and are you a JS-head
>>> that likes to help out getting the rest of the test suite to pass
>>> (basically
>>> only grouped reduce hasn't been ported from jscouch yet) please
>>> speak up so we can organize the effort.
>>>
>>> If you like temp views, I'd like to take your concerns into
>>> consideration.
>>>
>>>
>>> RIP Temp View?
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
>

Mime
View raw message