Hey Martin sorry for taking so long to respond to you especially when
you responded so quickly and thoroughly.  More in line ...

On 9/21/07, Martin Alderson <equim@planetquake.com> wrote:
Hi Alex,

For my fairly limited use mitosis is performing very well.

This is excellent news.

I've slacked off a bit over the last few weeks but will try to get back
onto it soon.

No worries me too.  I keep context switching these days myself.

For stable use there is still that critical issue where a change on the
same entry on multiple servers will lead to a permanently inconsistent
state.  I'll make that my priority soon.

Ok what I want to do is setup and environment soon and start trying to
reproduce this as well as any other problems.  This way I can work with
you or at least help answer any questions you have while attacking this
problem.  I guess we can have intermediate states where the replicas
are not in sync but they must converge over time so this is a critical issue.
I'll try to help out as best as I can on this.

I also recently came across a minor timeout issue which I think is a
problem in MINA.  I'll investigate that more soon too.

This is new to me but if you localize it I'm sure we can fix it.

Aside from that there are niggling issues like replication of schema

Yes this is something I am dreading. It's almost as if replicating a
partition must force replicating the schema across that cluster but
other clusters on different partitions will be all tied together for schema

This is why Microsoft decided to put schema into each partition for ADAM. 
Although painful we could do the same but I think we can work around this without
one offs but it will shift the design a bit.

Once these issues have been worked out I think we have a
stable replication system.


There are some features that I guess we really need to get in for it to
actually be useful for the majority of users though.  The main things
that come to mind are selective replication, encryption, schedules and
removing the dependency on Derby.

Yes these are also my concerns especially encrypting the replication channel.
We have some ideas that Ersin and I tossed around regarding all these issues.

I guess we should start simple.  First we should think about integrating the
UUID capabilities which are part of mitosis into ApacheDS core so UUID is
supported whether you turn on replication or not.

Then from there I'd like to make the quartz scheduler a core service (not
interceptor) that is accessible from the DirectoryService interface to be able to
schedule anything.

So moving these things up simplifies Mitosis a bit.

Then it's reasonable to just move the replication log off of Derby into a custom
store implementation based on JDBM.  Eventually we'll need to expose this data
via LDAP but we don't need to do it immediately.  For now getting rid of the dep
and having a clean store implementation is enough.

I know you and Ersin are thinking about possible design changes although
to me those can be done incrementally later on as required.  I doubt
that I will be a driving force behind these changes but am willing to help.

Right I agree with you on the incremental changes.  Don't sell yourself short the
new design changes are things I think you can easily grok.  Plus we're not moving
fast with anything at the moment - we just have ideas.  The problem has been
where to dig in and start getting traction.  I think we can do the things above in
parallel with introducing design changes as well.