On Tue, Jul 17, 2012 at 1:28 AM, Alex Karasulu <akarasulu@apache.org> wrote:


On Mon, Jul 16, 2012 at 6:50 PM, Emmanuel Lécharny <elecharny@gmail.com> wrote:

      - But i'm not against on replicating configuration completely. Some
one

     would want to simply clone the server in its entirety. ( This is not
     touched by RFC as i see ) Because our OSGI distribution is Karaf
based, we
     can use Karaf Cellar here. Cellar is used to keep multiple Karaf
instances
     synched in terms of Bundles,Features, and Configuration.(We can't use
its
     configuration aspects because we're keeping our component
configurations in
     ApacheDS itself). I quickly looked through its functionality and
code. It's
     incomplete but promising. Most importantly it provides API to initiate
     Karaf Instance replication.(Again not complete). So when replication
system
     is revisited we can make use of Cellar.
     - I looked at Apache ZooKeeeper a bit, and we should definetely
leverage

     it in our LDAP replication system. It is simply a distributed data and
     notification system for clusters. It has strength in node management
in
     clusters and good for implementing infrastructure related parts of
     replication system.

So i see no threat to OSGI from LDAP Replication. However i believe
ou=config should be excluded from replication in any scenario.

Not sure.

Not completely as i say, they must be controlled through other repliccation
system. I say they shouldn't be managed by LDAP replication, because as i
said: 2 server having the same configuration are not have to have same
runtime behaviour because of the different bundle configurations.
Two servers can't have the same configuration, otherwise we have a problem !

There are very few informations that are not to be replicated, because they belongs to the local server (the three I mentionned : IP, port, and name). That means we need to find a better way to store those informations. And if we store them in the DIT, then this part of the DIT must *not* be replicated.


Partial and fractional replication will come in handy here where we say this attribute or this entry does not get replicated and it does not. That simple. 
Yeah that is simple to implement. The important thing that matters here is that, just replicating configuration means nothing. Lets say one ApacheDS instance have installed some custom partition bundle and have a configuration entry pointing to that custom partition implementation. So replicating that configuration to the some other ApacheDS instance will not magically make this custom partition available unless we implement that magic into replication procedure. I think we can use Karaf-Cellar here if it becomes stable enough for us to use when we review replication system. If we can't use Cellar than we can simply describe how to replicate configuration between 2 servers, pointing out the importance of syncing 2 server in terms of installed bundles. No problem here !
 
If this is what you were thinking of, then yes, I'm on the same page. May be we should put thos information somewhere else than on the ou=config sub tree (root DSE ?). We could specifically address those points.


That's not something I'm feeling good about. We're going to change how we laid down the organization of configurations just for replication? Replication needs to advance to take into account these considerations instead of us injecting one offs to handle these replication scenarios.
Replication must be smart of course, through use of replication flags we don't have to store instance local properties in some other place. Because i believe there will be more local properties other than ip/port pair in the future. And the same local property type can be appeared in configuration at many place under different names, like kdc-server-port, ldap-server-port. So Replication flags must be designed for future flexibility. Tweaking the configuration layout does not help here.
 
I was pretty much thinking that we could store those informations in a plain text file, but that would be a bit overkilling, when we can store them in a the DIT too. Maybe storing those information sinto the ou=config entry could be the right thing to do, assuming that the ou=config is not replicated (we will only replicate what's under ou=config, ie, its children)


Please please please let's not fuck with this. This is the worst idea I've heard of yet. We don't need another one off here.
Actually i don't see that much harm here. I also once thought about how it would be to use configurations on plain-text files (OSGI Standart for so much framework and component). Though, i don't know historical decisions those led to this current configuration system. Maybe you can explain me why it's that fragile?

Current OSGI layer's code have a lot of code to deal with configuration kept in and tracked from ApacheDS Partition. It has pros and cons of course.(pros outweight cons IMO) And i like the way we can tweak the server from DIT. So I have no plan to change the way we store configuration.
 



    I believe if
we're gonna provide some way to replicate some ApacheDS instance's runtime
layout, we should not do it by LDAP Replication. Because :

     - It is not just config partition manages the server anymore.
     - We just can't be sure that 2 server having the same configuration
will

     have same runtime behavior, because they might have different
composition
     of bundles.
     - Replicating ou=config is hell of a job because of site specific
     configuration parameters.

Unless the server can detect which configuration applies to itself.

Bottom line, there are a very litte set of information a server needs to
keep local :
- its IP address
- its port
- its name

The name will be used to identify the configuration that applies to the
local server.
I don't believe simply keeping those 3 information local will help to make
replicating configuration information across multiple nodes consistent.
When thinking in terms of some system, there will be always some more local
configuration points other than IP/port/hostname IMO.
See upper with the suggestion I made about using pu=config entry but not replicating it. We could even extend the ObjectClass used to store the 3 pieces of informations to store more, if needed. ATM, though, this is all what we need locally. Everything else can be safely replicated, assuming that the code that reads and handle the configuration knows that it must be combined with the logal configuration.



--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com




--
Best Regards,
-- Alex