incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Troy Kruthoff <tkruth...@blit.com>
Subject Re: Views using JSON Arrays
Date Wed, 09 Jul 2008 17:43:29 GMT
Until this feature is implemented we are planning to put memcached in  
the middle, which supports a multi-key get.  This will not help much  
for hitting the views with startkey, etc, but for fetching multiple  
docs by id, it would appear to do the trick (once the docs where cached)

-- troy


On Jul 9, 2008, at 8:07 AM, Bradford Winfrey wrote:

> I'm going to wave the white flag on the multi-key search patch.  If  
> anyone else is feeling saucy enough to tackle it, feel free to, I'm  
> completely un-qualified to do so.  Thanks for all the input, but for  
> now I'm just going to have to execute the queries one at a time and  
> lump them together.  I'd love to investigate it further, but time  
> just doesn't permit it at the moment.  Maybe we could open a ticket  
> for it if enough people find it useful?  That's my two cents.   
> Thanks again - especially Paul!
>
>
>
> ----- Original Message ----
> From: Bradford Winfrey <bradswinfrey@yahoo.com>
> To: couchdb-user@incubator.apache.org
> Sent: Tuesday, July 8, 2008 3:14:56 PM
> Subject: Re: Views using JSON Arrays
>
> Oy, I've got to be way off then as to where I'm looking (in  
> couch_view.erl, notably process_doc and view_insert_query_results).   
> But yes you are correct in saying:
>
> "Where's the logic that's collecting json objects when they
> match the view parameter `key` or is between `startkey` and `endkey`?"
>
> I'm going to mull over this for a bit I think I finally made the  
> connection with your statement about how there is no real logical  
> "if-statement".  Heh, again of course unless someone chimes in and  
> says otherwise.
>
> Thanks again for the input.
>
> Brad
>
>
>
> ----- Original Message ----
> From: Paul Davis <paul.joseph.davis@gmail.com>
> To: couchdb-user@incubator.apache.org
> Sent: Tuesday, July 8, 2008 3:01:44 PM
> Subject: Re: Views using JSON Arrays
>
> When you say matching the key with what's being emitted, do you mean
> to ask, "Where's the logic that's collecting json objects when they
> match the view parameter `key` or is between `startkey` and `endkey`?"
>
> If so, that's all in make_view_fold_fun. Specifically it's
> PassedEndFun and how the result is used in [Neg|Pos]CountFun.
>
> The basic idea is that you have `startkey` and `endkey` (When `key` is
> provided `startkey` == `endkey` == `key`) and then you stream through
> the nodes in the view index starting at `startkey` and stream until a
> key is found that is beyond `endkey` which is the termination
> condition. The key is to remember that all docs in the view are
> ordered by key and that when you fold over them they are passed to the
> functions in make_view_fold_fun in order. Hence why there's no logic
> equivalent to "if record between start and end, add to results, else
> skip" (which is why its freaky fast). Its slightly more complicated to
> account for the reverse sorting, count, etc, parameters.
>
> At least, I'm about 95% sure that's how its working. We'll see if
> Damien laughs at me.
>
> Paul
>
> On Tue, Jul 8, 2008 at 2:25 PM, Bradford Winfrey <bradswinfrey@yahoo.com 
> > wrote:
>> I'm trying to find the function that actually does the matching of  
>> the key with what is being emitted, anyone have any insight where  
>> that is?  I have the multiple keys coming in, that was trivial, but  
>> now I'm trying to see how in fact the view indexes can do multiple  
>> lookups at once.  I must admit, I'm operating on no more than maybe  
>> 2 days of looking at Erlang so any extra pointers are welcomed.
>>
>>
>>
>> ----- Original Message ----
>> From: Damien Katz <damienkatz@gmail.com>
>> To: couchdb-user@incubator.apache.org
>> Sent: Thursday, July 3, 2008 1:51:38 PM
>> Subject: Re: Views using JSON Arrays
>>
>>
>> On Jul 3, 2008, at 2:34 PM, Jan Lehnardt wrote:
>>
>>>
>>> On Jul 3, 2008, at 20:17, Jan Lehnardt wrote:
>>>
>>>>
>>>> On Jul 3, 2008, at 20:07, Paul Bonser wrote:
>>>>
>>>>> On Thu, Jul 3, 2008 at 8:01 AM, Bradford Winfrey <bradswinfrey@yahoo.com
>>>>>> wrote:
>>>>>
>>>>>> We would get back each one of our tags as a key, yea?  Only if we
>>>>>> supplied one at a time.  So how does one go about supplying a
>>>>>> range, array (not sure what we'd call it here) of keys to be
>>>>>> searched on?  http://...?key=["octopus","hockey"] maybe?  I'm
>>>>>> unsure of the plan of attack for such a thing.  Maybe I'm just
>>>>>> going about it in the wrong direction.  Any thoughts?
>>>>>
>>>>> Hey everyone. I've pretty much just been watching the mailing list
>>>>> here myself.
>>>>>
>>>>> This is the second time somebody has asked about this since I've
>>>>> started watching this mailing list. I could see it being a  
>>>>> valuable
>>>>> feature.
>>>>>
>>>>> Would it be valuable to add the ability to query a view with
>>>>> multiple
>>>>> keys, getting the results for each of those keys back? Something
>>>>> like
>>>>> http://...?key_list=["octopus","hockey"]
>>>>>
>>>>> If I were interested in investigating how hard it would be to add
>>>>> that
>>>>> feature, where would I start looking in the code?
>>>>
>>>> This simply doesn't fit the way views work. They are single indexes
>>>> that you
>>>> can use for single-key or or key-range lookups. if you need random
>>>> access
>>>> you'd basically need to query the view as many times as you have  
>>>> keys
>>>> to search. No matter if you do that in front or behind the API.
>>>
>>> Hey you smart-ass, on the one hand you are trying to recruit new
>>> developers and on the other hand you discourage people from adding
>>> useful features. Make up your mind man!
>>>
>>> Paul, if you come up with a nice solution, we'd be happy to look at
>>> integrating it. Please give it a try. To investigate the code it is
>>> probably
>>> best to start at src/couchdb/couch_http.erl as it defines the HTTP  
>>> API
>>> and handles the requests you already know and just drill down from
>>> there.
>>
>> :)
>>
>> The view indexes can indeed do multiple lookups at once. The only
>> thing is I don't think a GET with all the keys are URL args is the  
>> way
>> to go as it can get really long and cause problems with proxies and
>> HTTP libs, a POST with the lookup keys as the body is probably  
>> better.
>>
>>
>>>
>>>
>>> Cheers
>>> Jan
>>> --
>>
>>
>>
>
>


Mime
View raw message