couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Hayton <theb...@purplebulldog.com>
Subject Re: multiview on github
Date Sat, 18 Sep 2010 01:00:31 GMT
I want to use it!  I just haven't gotten around to it.  I was going to try
and test it out this weekend and if I am able, I will certainly report back
what I find.

James

On Fri, Sep 17, 2010 at 5:55 PM, Chris Anderson <jchris@apache.org> wrote:

> On Mon, Aug 30, 2010 at 10:58 AM, Norman Barker <norman.barker@gmail.com>
> wrote:
> > Bob,
> >
> > I can and have been testing the multiview at this scale, it is ok
> > (fast enough), but I think being able to test inclusion of a document
> > id in a view without having to loop would be a considerable speed
> > improvement. If you have any ideas let me know.
> >
>
> I just want to bump this thread, as I think this is a useful feature.
> I don't expect to be able to test it in the coming weeks, but if I did
> I would. Is anyone besides Norman using this? Has anyone used it at
> scale?
>
> Cheers,
> Chris
>
> > thanks,
> >
> > Norman
> >
> > On Mon, Aug 30, 2010 at 10:49 AM, Robert Newson <robert.newson@gmail.com>
> wrote:
> >> I'm sorry, I've had no time to play with this at scale.
> >>
> >> On Mon, Aug 30, 2010 at 5:35 PM, Norman Barker <norman.barker@gmail.com>
> wrote:
> >>> Hi,
> >>>
> >>> are there any more comments on this, if not can you describe the
> >>> process (in particular how to obtain a wiki and jira account for
> >>> couchdb which I have been unable to do) and I will start documenting
> >>> this so we can put this into the trunk.
> >>>
> >>> Bob, were you able to do any more testing with large views, are there
> >>> any suggestions on how to speed up the document id inclusion test as
> >>> described below?
> >>>
> >>> thanks,
> >>>
> >>> Norman
> >>>
> >>> On Mon, Aug 23, 2010 at 9:22 AM, Norman Barker <
> norman.barker@gmail.com> wrote:
> >>>> Bob,
> >>>>
> >>>> thanks for the feedback and for taking a look at the code. Guidelines
> >>>> on when to use a supervisor within couchdb with a gen_server would be
> >>>> appreciated, currently I have a supervisor and a gen_server, but if
> >>>> couchdb has a supervision process I could remove that layer.
> >>>>
> >>>> I think plugins is a great idea, however intersection of views is such
> >>>> as common request, perhaps there needs to plugin system and if a
> >>>> plugin is rated enough it goes into trunk as a core feature.
> >>>>
> >>>> the four (or slightly more) summary is here
> >>>>
> >>>>
> http://github.com/normanb/couchdb/raw/trunk/src/couchdb/couch_query_ring.erl
> >>>>
> >>>> %
> >>>> % send an id from the start list to the next node in the ring, if the
> >>>> id is in adjacent node then the this node sends to the next ring node
> >>>> ....
> >>>> % if the id gets all round the ring and back to the start node then
is
> >>>> has intersected all queries and should be included. The nodes in the
> >>>> ring
> >>>> % should be sorted in size from small to large for this to be
> effective
> >>>> %
> >>>> % In addition send the initial id list round in parallel
> >>>>
> >>>> it really needs some eyes from the core couchdb coders to see how to
> >>>> speed up the inclusion testing, looping is bad even if it is done in
> >>>> parallel.
> >>>>
> >>>> Multiview is usable, I am using it with some pretty big mega-views (as
> >>>> per the raindrop) model, I am also available to add features to this
> >>>> as this is core part of our work and we want to give it to couch as
a
> >>>> contribution.
> >>>>
> >>>> thanks,
> >>>>
> >>>> Norman
> >>>>
> >>>> On Mon, Aug 23, 2010 at 5:05 AM, Robert Dionne
> >>>> <dionne@dionne-associates.com> wrote:
> >>>>> Hi Norman,
> >>>>>
> >>>>>  I took a peek at multiview. I haven't followed this too closely
on
> the mailing list but this is *view intersection*? Is there a 5 line summary
> of what this does somewhere?
> >>>>>
> >>>>>  I'm curious as to why the daemon needs to be a supervisor, most
if
> not all of the other daemons are gen_servers. OTP allows this but I think
> this is a good area where some CouchDB guidelines on plugins would apply.
> >>>>>
> >>>>>  It strikes me that views, the use of map/reduce, etc. are one of
the
> trickier aspects of using CouchDB, particularly for new users coming from
> the SQL world. People are also reporting issues with performance of views, I
> guess often because reduce functions go out of control.
> >>>>>
> >>>>>  I think the project would be better served if features like this
> were available as plugins. I would put GeoCouch in the same category. Its
> very neat and timely (given everyone wants to know where everyone else is
> using their telephone but without talking other than asynchronously), but a
> server plugin architecture that would allow this to be done cleanly should
> come first.
> >>>>>
> >>>>>  This is just my opinion. I'd love to see some of the project
> founders and committers weigh in on this and set some direction.
> >>>>>
> >>>>> Best regards,
> >>>>>
> >>>>> Bob
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Aug 22, 2010, at 5:45 PM, Norman Barker wrote:
> >>>>>
> >>>>>> I would like to take this multiview code and have it added to
trunk
> if
> >>>>>> possible, what are the next steps?
> >>>>>>
> >>>>>> thanks,
> >>>>>>
> >>>>>> Norman
> >>>>>>
> >>>>>> On Wed, Aug 18, 2010 at 11:44 AM, Norman Barker <
> norman.barker@gmail.com> wrote:
> >>>>>>> I have made
> >>>>>>>
> >>>>>>> http://github.com/normanb/couchdb
> >>>>>>>
> >>>>>>> which is a fork of the latest couchdb trunk with the multiview
code
> >>>>>>> and tests added.
> >>>>>>>
> >>>>>>> If geocouch is available then it can still be used.
> >>>>>>>
> >>>>>>> There are a couple of questions about the multiview on the
user
> /dev
> >>>>>>> list so I will be adding some more test cases during today.
> >>>>>>>
> >>>>>>> thanks,
> >>>>>>>
> >>>>>>> Norman
> >>>>>>>
> >>>>>>> On Tue, Aug 17, 2010 at 9:23 PM, Norman Barker <
> norman.barker@gmail.com> wrote:
> >>>>>>>> this is possible, I forked geocouch since I use it,
but I have
> already
> >>>>>>>> separated the geocouch dependencies from the trunk.
> >>>>>>>>
> >>>>>>>> I can do this tomorrow, certainly be interested in any
feedback.
> >>>>>>>>
> >>>>>>>> thanks,
> >>>>>>>>
> >>>>>>>> Norman
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Aug 17, 2010 at 7:49 PM, Volker Mische <
> volker.mische@gmail.com> wrote:
> >>>>>>>>> On 08/18/2010 03:26 AM, J Chris Anderson wrote:
> >>>>>>>>>>
> >>>>>>>>>> On Aug 16, 2010, at 4:38 PM, Norman Barker wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> I have made the changes as recommended,
adding a test case
> >>>>>>>>>>> multiview.js and also adding the userCtx
to open the db.
> >>>>>>>>>>>
> >>>>>>>>>>> I have also forked geocouch and this is
available here
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> this patch seems important (especially as people
are already
> asking for
> >>>>>>>>>> help using it on user@)
> >>>>>>>>>>
> >>>>>>>>>> to get it committed, it either must remove the
dependency on
> GeoCouch, or
> >>>>>>>>>> become part of CouchDB when (and if) GeoCouch
becomes part of
> CouchDB.
> >>>>>>>>>>
> >>>>>>>>>> Is it possible / useful to make a version that
doesn't use
> GeoCouch? And
> >>>>>>>>>> then to make the GeoCouch capabilities part
GeoCouch for now?
> >>>>>>>>>>
> >>>>>>>>>> Chris
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Hi Norman,
> >>>>>>>>>
> >>>>>>>>> if the patch is ready for trunk, I'd be happy to
move the
> GeoCouch bits to
> >>>>>>>>> GeoCouch itself (as GeoCouch isn't ready for trunk
yet).
> >>>>>>>>>
> >>>>>>>>> Lately I haven't been that responsive when it comes
to GeoCouch,
> but that
> >>>>>>>>> will change (in about a month) after holidays and
FOSS4G.
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>>  Volker
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message