incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Barker <norman.bar...@gmail.com>
Subject Re: multiview on github
Date Mon, 23 Aug 2010 03:35:18 GMT
that should be 'couchdb should not be in version control', sorry not
used to git.

On Sun, Aug 22, 2010 at 9:22 PM, Norman Barker <norman.barker@gmail.com> wrote:
> Bob,
>
> I am testing on 10000+ documents, I appreciate that we need to
> establish when a multi-process as opposed to a tbd (suggestions
> welcome) approach is required. The startkey / endkey is an issue
> though, is there a better way to test inclusion?
>
> The speed of the multiview is directly linked to the size of the
> smallest view result though, so total documents isn't a factor.
>
> I am still thinking about fti, I am testing with clucene, but the
> external handler problem is the same, how to make it stream in order.
>
> I will fix the local_dev.ini problem tomorrow, couchdb should be in
> version control.
>
> Any hints on how to test inclusion are appreciated, it will greatly
> speed up collation.
>
> thanks,
>
> Norman
>
>
>
> On Sun, Aug 22, 2010 at 4:15 PM, Robert Newson <robert.newson@gmail.com> wrote:
>> I'm concerned about the performance of this on non-trivial databases,
>> given the iteration of all items between startkey and endkey. I don't
>> have time to test it this week but I'd be interested to hear the time
>> it took to do a multiview on two views of, say, a million rows each
>> (especially as compared to the two normal view calls).
>>
>> I was also intrigued to see the code handles fti too, a problem I have
>> spent some time thinking about without finding a satisfactorily
>> performant solution too. I note that, as written, it doesn't appear to
>> work because the fti call (I'm assuming couchdb-lucene) will only
>> return the top N matching hits, so at best you can filter those
>> against another view (perhaps that's useful?). The trick to merging a
>> view and an fti result together would be to get the results from both
>> in the same order and step through the rows, filtering as you go.
>> Sorting in Lucene has a large memory hit so I gave up on that
>> solution.
>>
>> Finally, your patch appears to add two generated files (local_dev.ini
>> and etc/init.d/couchdb) to the branch which should be fixed (add your
>> settings to default.init.tpl.in instead).
>>
>> I should end by saying that if the problems above can be solved then
>> this would be a very useful addition to CouchDB and one that is
>> frequently requested. It might also be a model for multi-machine
>> views.
>>
>> B.
>>
>> On Sun, Aug 22, 2010 at 10:45 PM, Norman Barker <norman.barker@gmail.com> wrote:
>>> I would like to take this multiview code and have it added to trunk if
>>> possible, what are the next steps?
>>>
>>> thanks,
>>>
>>> Norman
>>>
>>> On Wed, Aug 18, 2010 at 11:44 AM, Norman Barker <norman.barker@gmail.com>
wrote:
>>>> I have made
>>>>
>>>> http://github.com/normanb/couchdb
>>>>
>>>> which is a fork of the latest couchdb trunk with the multiview code
>>>> and tests added.
>>>>
>>>> If geocouch is available then it can still be used.
>>>>
>>>> There are a couple of questions about the multiview on the user /dev
>>>> list so I will be adding some more test cases during today.
>>>>
>>>> thanks,
>>>>
>>>> Norman
>>>>
>>>> On Tue, Aug 17, 2010 at 9:23 PM, Norman Barker <norman.barker@gmail.com>
wrote:
>>>>> this is possible, I forked geocouch since I use it, but I have already
>>>>> separated the geocouch dependencies from the trunk.
>>>>>
>>>>> I can do this tomorrow, certainly be interested in any feedback.
>>>>>
>>>>> thanks,
>>>>>
>>>>> Norman
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Aug 17, 2010 at 7:49 PM, Volker Mische <volker.mische@gmail.com>
wrote:
>>>>>> On 08/18/2010 03:26 AM, J Chris Anderson wrote:
>>>>>>>
>>>>>>> On Aug 16, 2010, at 4:38 PM, Norman Barker wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have made the changes as recommended, adding a test case
>>>>>>>> multiview.js and also adding the userCtx to open the db.
>>>>>>>>
>>>>>>>> I have also forked geocouch and this is available here
>>>>>>>>
>>>>>>>
>>>>>>> this patch seems important (especially as people are already
asking for
>>>>>>> help using it on user@)
>>>>>>>
>>>>>>> to get it committed, it either must remove the dependency on
GeoCouch, or
>>>>>>> become part of CouchDB when (and if) GeoCouch becomes part of
CouchDB.
>>>>>>>
>>>>>>> Is it possible / useful to make a version that doesn't use GeoCouch?
And
>>>>>>> then to make the GeoCouch capabilities part GeoCouch for now?
>>>>>>>
>>>>>>> Chris
>>>>>>>
>>>>>>
>>>>>> Hi Norman,
>>>>>>
>>>>>> if the patch is ready for trunk, I'd be happy to move the GeoCouch
bits to
>>>>>> GeoCouch itself (as GeoCouch isn't ready for trunk yet).
>>>>>>
>>>>>> Lately I haven't been that responsive when it comes to GeoCouch,
but that
>>>>>> will change (in about a month) after holidays and FOSS4G.
>>>>>>
>>>>>> Cheers,
>>>>>>  Volker
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message