couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From afshin afzali <a.afzali2...@gmail.com>
Subject Re: querying multiple views
Date Sun, 11 Jul 2010 17:13:26 GMT
Thank You Very Much Guys.

-- afshin

On Sun, Jul 11, 2010 at 7:27 PM, J Chris Anderson <jchris@gmail.com> wrote:

>
> On Jul 11, 2010, at 7:27 AM, Norman Barker wrote:
>
> > Afshin
> >
> > I have got the all clear from my work to release this as a patch, I
> > expect to be putting something up on github by the end of the week
> > (internal paperwork permitting). I am going to implement it as an
> > external handler so it can be used and reviewed and from there it will
> > be under a do what you want with it license so it can go into couchdb
> > if accepted.
> >
>
> If you're implementing in Erlang, you probably don't need to go full hog as
> an external.
>
> Because the configuration system is so modular, you are probably best
> adding it as a new httpd_design_handler, or httpd_db_handler.
>
> It should be easy to create a new module and link it in via the
> configuration file.
>
> I don't know if you plan to allow querying across databases (I'd suggest
> restricting the queries to a single database if you want to stay within
> CouchDB's security model, and have it more likely to be accepted as a
> patch.)
>
> We should really move this discussion to dev@ -- a lot of the developers
> only give a cursory glance at the user list, so you will get more valuable
> feedback there.
>
> Thanks for taking the time to write and release the patch!
>
> Chris
>
> > Chris, thanks for the help with the reduce function and confirming the
> concept!
> >
> > Norman
> >
> > On Sun, Jul 11, 2010 at 8:02 AM, afshin afzali <a.afzali2003@gmail.com>
> wrote:
> >> Hi Norman, Chris
> >>
> >> I just wanted to say this is the same problem we are currently facing
> >> with. We are implementing a Local Business Directory application on
> >> couchdb. Our searches need to combine several keys together to find
> >> right entries. To do something like that in server side, we had the
> >> paging mechanism problem, so we have chosen that do Norman's algorithm
> >> in client side! I'll appreciate if there will be a successful progress
> >> in this issue.
> >>
> >> BEST,
> >> -- afshin
> >>
> >> On 7/8/10, J Chris Anderson <jchris@apache.org> wrote:
> >>>
> >>> On Jul 8, 2010, at 10:43 AM, Norman Barker wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I have been thinking about how to query multiple views at one time.
> >>>>
> >>>> I have an erlang handler in couchdb that takes a http post containing
> >>>> N view queries, each query contains a startkey and an endkey, I then
> >>>> open up each view in parallel (using pmap) and accumulate the doc ids,
> >>>> then I use the erlang sets module to get the unique values. All good
> >>>> and looks pretty (and works), though it doesn't scale since I am
> >>>> holding all the results on the server (potential memory overload!)
> >>>> whereas I would like to stream the results to the client one by one.
> >>>>
> >>>> I am thinking of doing the following but have some questions;
> >>>>
> >>>> My first question is when I do
> >>>>
> >>>> couch_view:fold(View, FoldlFun, FoldAccInit,
> >>>> couch_httpd_view:make_key_options(Args)),
> >>>>
> >>>> is there a way to call the _count reduce function in code to find the
> >>>> number of rows in the slice between startkey and endkey?
> >>>>
> >>>> If so, I would like to order all the views in the posted query
> >>>> document by the result of _count from smallest to largest.
> >>>>
> >>>> I would then fold over the smallest result view and pull each document
> >>>> id (*) in turn.
> >>>>
> >>>> With each document id I would then call each of the other views in
> >>>> turn with their startkey and endkey and in addition include
> >>>> startkey_docid and endkey_docid with the docid in * above, again
> >>>> calling _count I can test for inclusion. If the doc id is in all views
> >>>> then I will immediate stream this to the client.
> >>>>
> >>>> Am I doing something stupid, is this optimal?
> >>>>
> >>>
> >>> It sounds like you are on the right track. this could be a very
> valuable
> >>> patch to CouchDB once you have it working.
> >>>
> >>>> Any help with the programmatic _count call would be great.
> >>>>
> >>>
> >>> One hint: maybe the call to reduce_to_count will help.
> >>>
> >>> Here's an implementation of a reduce query in Erlang.
> >>>
> >>> http://github.com/jchris/hovercraft/blob/master/hovercraft.erl#L217
> >>>
> >>> Sorry I can't be more helpful. I've successfully bootstrapped this
> stuff in
> >>> my head before, but it always takes a couple of hours of turning my
> brain
> >>> into a step debugger.
> >>>
> >>> Good luck!
> >>>
> >>> Once you get deeper into the code you might have better luck getting
> >>> responses on the dev@ list or maybe the #couchdb IRC channel on
> freenode.
> >>>
> >>> Chris
> >>>
> >>>
> >>>> thanks,
> >>>>
> >>>> Norman
> >>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message