incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Buisson <guillau...@doremitechno.org>
Subject Re: Ad-hoc filtering
Date Tue, 26 Feb 2013 15:18:39 GMT
Docs for lists are available here:
http://wiki.apache.org/couchdb/Formatting_with_Show_and_List

I used this method to do basic "on the fly" filters and pagination,
The list acts as some kind of decorator for view, my feeling is that it's
been implemented to do format conversions, but you can use it for other
things...
the req var contains all request arguments (headers, search etc.), it is
then easy to test each rows/keys and only return the corresponding docs.

This will however come at a cost, the way i see it, all data from a view is
passed to the list and evaluated on request with no "caching strategy",
if your view fetches thousands of docs, this can be very ressource
hungry...


2013/2/26 Guillaume Buisson <guillaumeb@doremitechno.org>

> Why don't you use couchDB lists ?
>
> On lists you can fetch search parameters and filter data accordingly,
> i know this is not optimal performance wise, but still it's better than
> pushing views each time.
>
>
> 2013/2/26 Jan Lehnardt <jan@apache.org>
>
>>
>> On Feb 26, 2013, at 14:54 , Troy Farrell <troy@entheossoft.com> wrote:
>>
>> > It's probably not a popular opinion, but this application does not sound
>> > well-suited for CouchDB.  If you are creating views all the time, using
>> > them once and still downloading all the data for post-processing, then
>> > something is clearly wrong.  We may be able to help you restructure the
>> > data and views so you need fewer of them, but if you truly need lots of
>> > dynamic queries, CouchDB may not be the best choice for your
>> application.
>> > (I would love for someone on this list to tell me I'm wrong.)
>>
>> I think determining when CouchDB is *not* a good fit is crucial. I for one
>> am not interested in unhappy users.
>>
>> Best
>> Jan
>> --
>>
>>
>> >
>> > Troy
>> >
>> > On Mon, Feb 25, 2013 at 5:50 PM, Daniel Gonzalez <gonvaled@gonvaled.com
>> >wrote:
>> >
>> >> Hi,
>> >>
>> >> It is not possible for me to define and manage all possible views that
>> my
>> >> application needs. Each time that I need to access data with a new
>> >> filtering mechanism I have to:
>> >>
>> >>   1. Check if that view, or a similar one, already exists. (How would I
>> >>   call this if I had created this half a year ago?)
>> >>   2. If it doesn't, I must create it (javascript, not my piece of
>> cake. I
>> >>   am most of the time in python land).
>> >>   3. I have to push the new design doc to all my servers (did you also
>> >>   push to that-server-that-you-never-use? Do you really need to push
>> this
>> >>   view to that server? I don't remember, really.)
>> >>   4. I have to trigger the new view (which can take some time in
>> databases
>> >>   with lots of docs), in all servers.
>> >>
>> >> So now you need to filter for False instead of True? Great! Rinse and
>> >> repeat! At the end of the day I end up with dozens of views, some of
>> them
>> >> outdated, which I am not using anymore, but I am unable to maintain.
>> >>
>> >> Besides, defining lots of views becomes impractical: the more views you
>> >> have in a design doc, the longer it takes a refresh when the design
>> doc is
>> >> modified. And the more views, the more space the database uses.
>> >>
>> >> At some point this becomes very painful. Which means that for some
>> data I
>> >> just do client-filtering.
>> >>
>> >> Very often I must perform the following operations:
>> >>
>> >>   1. Access a view
>> >>   2. Filter the documents in the view according to some criteria
>> (which is
>> >>   not supported by a view)
>> >>   3. Paginate the resulting documents
>> >>
>> >> Since the filtering is done in the client, I am forced to get all
>> >> documents. I will discard most of them, but since I do not know which
>> ones,
>> >> I still must get them all. This is very slow.
>> >>
>> >> Since for some data I do view-filtering, and for other data I do
>> >> client-filtering, some operations are taking a very long time to
>> complete,
>> >> whereas others are completed very fast. Which is confusing for the
>> user.
>> >>
>> >> It would be great if couchdb supported field filtering. For example I
>> would
>> >> pass a dictionary like this: { 'available' : true, 'user' : 'user1' }
>> And I
>> >> would get only the documents which satisfy the filter. Using a single
>> view.
>> >> A simple "is equal to" operation for the given fields would go a
>> looong way
>> >> to solve most of my filtering needs.
>> >>
>> >> The view engine would perform the standard indexing, and the filtering
>> >> would be an extra layer (slowing things down a bit, probably). It
>> would be
>> >> a compromise between view indexing and client filtering.
>> >>
>> >> Does this mechanism already exist? Does it make sense? Would it ever be
>> >> implemented?
>> >>
>> >> Thanks
>> >> Daniel Gonzalez
>> >>
>>
>>
>

This email has been verified by Google Postini filtering system 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message