couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From J Chris Anderson <jch...@gmail.com>
Subject Re: Picking first free document from database
Date Fri, 04 Jun 2010 16:24:59 GMT

On Jun 4, 2010, at 3:40 AM, Attila Nagy wrote:

> Hi,
> 
> I've also figured out workarounds (like giving back multiple document IDs and randomize
them either on the client or -is it possible?- on server side), or pre-allocate (or somehow
hash) the document space for given clients (applications), but this doesn't solve the problem
of generating a lot of conflicts and the need for constantly burn server and client CPU time.
> 
> Should I assume from your e-mail that there is no solution for this problem currently
(apart from these hacks of course :)?
> 

Using update conflicts (which are as cheap as reads) to treat docs as work tickets is totally
valid in CouchDB. I'd avoid having clients attack the list in the exact same order all the
time (eg random is better), but I wouldn't worry too much about the cost of a few 409s.

> Thanks for the ideas.
> 
> On 06/04/10 12:31, Sebastian Cohnen wrote:
>> hmmm...
>> 
>> i would let the client fetch a available doc from couch and update it's status. in
case another process tries to update the document status in between couch will respond with
409 Conflict (since it cannot provide the correct mvcc token). if you get a conflict take
the next one. the load for the server should be okay, since AFAIK detecting conflicts and
rejecting updates is relatively cheap.
>> 
>> another idea, that pops into my mind: if you really have a ton of load/requests,
you could use some kind of proxy, which ensures that every document id is only given out one
time to once client. this proxy would be your new bottleneck, but you also could use some
queuing technique here and fetch batches of available docs from couch...
>> 
>> On 04.06.2010, at 11:57, Attila Nagy wrote:
>> 
>>   
>>> Hello,
>>> 
>>> I would like to build an application on CouchDB, which allocates free documents
to the users. Each document would have a status, which indicates whether it's available for
assigning, or not.
>>> Because of the great number of new users wanting available documents, I think
the default model of writing a view, which lists available documents, and picking one from
it (for efficiency limiting the number of return values to one and picking that) won't work.
The contention will make applications do a lot of conflicts, which slows down (or even make
it impossible) the new assignments, and is a lot of unnecessary load for both the server and
the client applications.
>>> 
>>> So I would need a method, which would pick the first document from the database
which has "available" in its status field, and immediately (atomically) set it to "unavailable",
and return the document ID to the caller.
>>> 
>>> I've found two posts, which do something similar (server side changes to documents):
>>> http://blog.couch.io/post/410290711/atomic-increments-in-couchdb
>>> http://wiki.apache.org/couchdb/Document_Update_Handlers
>>> 
>>> I'm very new to CouchDB, and I can't yet to see whether these can be adapted
to the above task (these two are tied to a given document).
>>> 
>>> So in short I would need: a server-side function, which traverses on all entries
(uses a view for efficiency?), picks the first one which has status: available, sets that
to unavailable and returns the docID to the client. And of course all of this atomically,
meaning no two calls to this method should give back the same docID.
>>> 
>>> Is this doable somehow with CouchDB?
>>> 
>>> Thanks,
>>>     
>>   
> 


Mime
View raw message