couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <dam...@apache.org>
Subject Re: I got a little carried away ...
Date Thu, 05 Mar 2009 01:58:04 GMT

On Mar 4, 2009, at 8:34 PM, Adam Kocoloski wrote:

> I was working through some replication tickets this week and  
> thinking more and more that the replicator could benefit from being  
> restructured along OTP principles.  So, I went ahead and did it.   
> Here's the structure I've worked out so far:
>
> http://github.com/kocolosk/couchdb/tree/otpify-replication
>
> * couch_primary_services gets a new child  
> (couch_replication_supervisor) which is itself a supervisor using a  
> one_for_one strategy.
>
> * Replications are modeled as gen_servers and added as transient  
> children of couch_rep_sup.  couch_rep_sup ensures that identical  
> replications do not run simultaneously.
>
> * The gen_server spawn_links a separate process that enumerates the  
> docs on the source and makes synchronous calls to the gen_server  
> telling it to copy the documents that don't already exist on the  
> target.
>
> * Opening and saving of documents and attachments is handled  
> internally by the gen_server.  I'm hoping that concentrating all  
> this activity in the server will help us do a better job of pipeline  
> and memory management.
>

This sounds good.


> Sorry for not discussing a change this big ahead of time.

We are discussing it now :)

> Damien, I saw that you had some recent checkins related to  
> replication security; I'd be happy to merge those in.  The  
> replication API itself has not changed at all.

Let me know if you need any help merging.

>
> This branch is still work-in-progress, but in my opinion it's almost  
> as fast as and more robust than the code in trunk right now.  If it  
> meets with peoples' approval and we can get some feedback from the  
> various reporters in JIRA, it might make sense to tag it for 0.9.   
> Cheers,

This work will also be useful for setting up continuos replications.  
Soon we'll have Comet events to notify a waiting replicator of changes  
to replicate, so change replicate as soon as they happen, not just on  
a schedule.

-Damien

Mime
View raw message