On Wed, Jan 5, 2011 at 11:18 PM, Emmanuel Lécharny <elecharny@apache.org> wrote:
On 1/5/11 9:49 PM, Alex Karasulu wrote:
On Wed, Jan 5, 2011 at 10:06 PM, Emmanuel Lécharny<elecharny@apache.org>wrote:

On 1/5/11 8:17 PM, Alex Karasulu wrote:

On Wed, Jan 5, 2011 at 8:13 PM, Emmanuel Lecharny<elecharny@gmail.com
 On 1/5/11 6:49 PM, Alex Karasulu wrote:
So when considering compatibility we have to consider several things
just APIs and SPIs:

  o Database Format
  o Schema
  o Replication Mechanism
  o Configuration
  o API Compatibility
  o Plugins - We have pseudo plugins like Partitions, Interceptors and
Handlers that users can alter which involve SPIs.

 I would get the Database Format and Configuration out of the equation.
up to us to provide tools to migrate from one format to the other. Don't
me wrong : when I say that configuration is out of the equation, I mean
the configuration can change, not its format (ie switching from XML to
is possible between to major releases, not between two minor releases).

 Will this be transparent to the user? Meaning can he just upgrade the
software and the migration will occur without any change in their
or anything noticeable in performance, wait time on startup? More

(1) Does the user have to run a tool to migrate from one version to the

Definitively, yes.

This is a bit worrisome to me but I cannot figure out why yet. Something in
my gut that I have not translated into real consequences yet.

I can see advantages with such a tool which allows us to change these
formats and configurations. But the disadvantage is the one off of having to
figure out if you need the tool with every minor or micro release. It's yet
another one off and the tool make take a day to run depending on how big the
DIB is.

We are not likely to change those DB formats often. We did it 3 times in 6 years, and it was for some major refactoring, I can't foresee any small modification that could be needed soon.

Yes that's probably the case especially after a GA and when we go MVCC that might alter the DB format, however MVCC would be another major release in itself.

However, I do think that at some point, this might be necessary to ofter such a tool. It's too easy to rely on our user to get their data exported and reimported using a LDIF file, as it has some consequence : the operational attributes will be modified.

If this is a low level partition tool then there might be ways around loosing things like timestamps and modifiers. 

However with modularity and OSGi these points become less problematic.

If this is set as the policy then this tool must always be provided. Those
who push this as the way then need to be held responsible for providing the
tool when needed. That sort of goes against the community dynamic: it's
going to be a must do for those accepting the policy.
Same opinion here.

So for those who want it, it should be provided by them on demand before any
release takes place. That's kind of harsh.

Instead if we respect the DB format and just release with the right
versioning schemes then we should be OK. If compatibility breaks then a
major release can be done and tools can still be provided to migrate
optionally without requirement.

See my point here?
yes. But again, such a modification is not likely to happen, is not part of the contract, and can be manage if we provide a migration tool. It's quite a common practice in the industry, and should not be a burden. It should even not require a major release, IMO.

I think it's debatable - again it's the contract with users. If this is going to be a project conducted with professionalism these kinds of policies are just as important to follow as test coverage or feature design. 2.0 is being targeted for the enterprise with the features put into it so things need to be tight.

We can't be relaxed about shifting database formats and configurations between point releases. 

 (2) If a user has 100 Million entries and there's a migration operation
running with this take say a few hours to start up the server?

This should be a low level tool, so it should act on the Backend interface

Yeah but it can still take days depending on the DB size but should not be
an issue with 90% of our users.
The day we have a user with 100 million entries, trust me, we will have other issues than just dealing with the migration of its database :)

It does not matter if we have other issues, that's besides the point. Regardless of other possible issues it will take about 2 days to migrate and that's a long time. Why also bother building tools if we follow some conventions?

Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu