directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <>
Subject Re: ADS 2.0 : what, how and when?
Date Wed, 05 Jan 2011 16:06:13 GMT
I will reply to both Kiran and Alex mail here.

Kiran :
"a small correction it is only master-slave replication atm" :

Hmm, too bad. We need MMR. I overlooked that then. The question is : can 
we do the MMR between RC1 and RC2 ?

>> What
>> ----
>> (FTR, The last release we did, Apache Directory Server 1.5.7 was issued on
>> april, 24th, 2010)
> Even if we release a 2.0-RC1 in the very near future say a couple weeks from
> now, I'd recommend releasing a 1.5.8 so the features and changes made since
> 1.5.7 can be tested by users. There have been some big changes especially on
> the Kerberos side.
> A release would take less than a weeks time (vote and all) and should not
> hold back the work being done to progress towards 2.0.

I have three concerns with approach :
- first, we agreed to let down the 1.5.8 last year. That's not a blocker 
- second, even if it takes one week to get the release done, it's still 
one week. Assuming we are close to 2.0 (ie, 4 to 8 weeks), then I don't 
think it worths losingthis week in an intermediate and short live release
- most important, if we release a 1.5.8, that will send a signal to 
users that they have to migrate to 1.5.8. Those who will do, and will 
see a 2.0 release a few weeks (months?) later may be stuck for a long 
time in such a release.

I'm not extracting this from thin air, we actually have one user, and 
not the least one (IBM Rational Directory Server) who is *stuck* in 
1.5.5, and they won't migrate to 1.5.8. They probably will jump to 2.0 
when out.

> Hopefully, we have JIRA which stores the issues, tasks and wishes, and it's
> the first input we will use to determinate the 'what'. Let's move back to
> the past a bit. In may, 26th, 2010, a mail about the version scheme was sent
> and discussed on the directory ML [1][2]. What was proposed back and agreed
> then was :
> - skip ADS-1.5.8 and target a 2.0.0-RC1 instead
> - skip shared-0.9.20 and target for a 1.0.0-RC1 instead
> - Keep the RC in order to give us the opportunity to stabilize the code
> *and* the documentation
> This might have been presumed thinking 2.0 was not far away. Now we're 8
> months out since the last release. So we might want to re-evaluate this, and
> you made some points about not releasing shared 1.0.0-RC1 to avoid having to
> solidify those API's for 2.0 because this might slow 2.0 down.
yes, because the API will have a *massive* impact on users. This is 
critical, and I share your vision about the stability requirement here.

I don't think that it's the same story for the server.
>> What was also raised at this time is that once we will jump for a
>> 2.0(server) and 1.0(api), we will be stuck with the exposed interfaces for a
>> very long time, so we'd better be careful. This still stand.
> Yeah this is my main worry. Let me though put this discussion out into
> another thread or two for better clarity and tracking.

We must distinguish between the LdapAPI (ie, merge of shared and Client 
API) and the server API (ie core-api plus the base classes in shared : 
entry, etc).

I'm less worried about the Server API than I'm for the LdapAPI.

Also I'd like to quote you on that aspect, because I find your post 
( :

"Also I remember a great email conversation almost 7 years ago on 
another project's mailing list. It was agreed that releasing fast and 
releasing often was more important than making sure the code was 
absolutely stable. Firstly because you can never really be absolutely 
stable and second because only life in the wild will show if the code is 
fit enough: fixing sometimes what you think is the big issue does not 
really fix the big problems in the wild. *Third and most important was 
the idea that stability is not that important. Yeah you heard me right.* 
Just get the feature out and let the user community report the problems 
so you can fix it fast but fix it fast and release fast. This keeps the 
user community more involved with the project and communicating with us. 
It's actually the best way to keep a vibrant community and spend less 
time testing/improving something which you really cannot tell is broken. 
You only know in the wild. I'm not saying write and release buggy 
software and nor were these guys saying this. I'm saying let's fear not 
providing stability on the first major release. If we blindly stick to 
the release early release often strategy and communicate with our users, 
then eventually we will converge towards stability with the least 
testing effort and the greatest gain in community. So let's make sure we 
get the releases out fast to fix the issues that arise for our users 
when they contact us with problems. This way we can stimulate a more 
vibrant user community while allowing us to focus more time in the right 
places developing rather than testing."

That does not mean we should disregard the Server API stability. We must 
do it, even if it postpone the 2.0 release for a few weeks.

We all agree that the "release often" mantra is a good one, but I don't 
think it applies well with the version scheme we are using. However, we 
decided to follow this versioning scheme, even if it sounds like a dead 
body we are carrying, and I think it's better delaying a long expected 
release for a few more weeks rather than delivering an intermediary 
version and to release a major version a few weeks later. It sounds not 
consistent to me.

>> We then started to move forward, cleaning some parts that needed it,
>> following the classic #bug report  :
>> - installers were totally rewrote [8]
>> - configuration has been completely re-factored, it's now stored in DIT
>> - all the 1.5.8/1.5.9 issues have been moved to 2.0.0-RC1 in JIRA
>> - performances of each operations were greatly improved (by a factor 2 to
>> 4) [5]
>> - some discussions about the JDBM backend occured, we agreed we should have
>> a MVCC backend sooner or later, but no clear decision was even stated about
>> when we should do that. However, recollecting my memory, it seems that it's
>> out of question for 2.0.
>> - Multi-Master replication has been added [2]
>> - we moved documentation from Confluence to use Dockbook instead [6]
>> - PasswordPolicy have been added
>> - the ACI subsystem has been reviewed [3]
>> - some refactoring of shared has been discussed and started [4]
>> - the various LDAP messages data structure have been merged (a long
>> expected task)
>> - some discussion started about the shared/API merge [7][9][10]
>> - the kerberos codec has been completely rewritten [11]
>> - AdministrativeModel implementation refactoring has been started (support
>> of IAP is at stake)
> Sounds to me like we should get a 1.5.8 release out to the community fast
> while continuing with 2.0. There's a lot of things that could use real work
> tire kicking here. It will not necessarily slow us down and it will get us a
> bit more tidy for 2.0. We're still going to have to tidy up as usual for
> 2.0-RC1.
See upper.

>> All those parts have been heavily discussed, as discussion about a 2.0
>> release started back in january 2009 ! [12] This effort has been pursued and
>> in may 2010, we started a bug parade [13] (reports are available on [14],
>> [15], [16], [17], [18]). Then we hit some serious issues in the code that
>> needed more than a few lines of code, bt some serious refactoring (namely,
>> config in DIT, ACI subsystem, Kerberos refactoring, AdministrativeModel).
>> Those huge refactoring stopped the release effort until they are done.
>> Right now, we are almost ready to start again to run the bug parade, once
>> the Administrative Model is finished (still working on that).
> Release early release often. If we get a 1.5.8 out and let peeps kick the
> tires but tell them, look 2.0 is coming soon so don't get hooked on this

See upper.

>> Sooo... time to review what's missing, and what's need some love now. This
>> is the 'What' part we all are looking at, and the best point to start it
>> will be to look at opened JIRAes. Otherwise, we still have to discuss some
>> serious re-factoring to be done in Shared.
>> Back in december, Alex started to review Shared, we discussed about it, and
>> we agreed that the work started on september was not enough. The rationals
>> are clear :
>> - shared and API *are* the same thing. We want to merge the client-API and
>> shared to provide a LdapAPI (and we even started a new project for that).
>> - many classes are useless, and need to be removed. Alex started to do that
>> in a shared branch
> Yeah I was doing some experiments in my branch not certain one way is better
> than another but after seeing some nice impact I can start solidifying a
> direction and express my findings here.

We are all waiting for some feedbacks. Those modifications are a direct 
follow up of the first round of modifications we have started back in 
september. We didn't finished, you picked up the torch, and that's good.
>> There may be some other elements, but nothing is yet decided, so it's fine
>> to add there what is missing.
>> Otherwise, we have to remember that we have 3 different 'release' at stake
>> here :
>> - ADS 2.0
>> - Studio 2.0
>> - LdapAPI 1.0 (ex-shared)
>> LdapAPI is absolutely *critical*. Once it's out, there is no way for us to
>> change it, because it will be heavily sued, as it's meant to be a
>> replacement for JNDI, and many of the outdated LDAP API. So we have to be
>> extraordinary cautious in this area.
> Exactomundo!!! This is my massive fear. There was immense work done here and
> it's awesome and can save the world from JNDI hell when dealing with LDAP.
> So we need to do it right and adhere to API maintenance conventions.
> Unfortunately though the LDAP Client API is not only restricted to the ldap
> client api module, the API's surface area spans across several shared
> modules: really almost all of them. So these API's don't have clear
> boundaries. I'll discuss this in another thread.

Sure. let's not hijack this long thread...
>> However, and that's the good point, we can spend as much time as needed to
>> get LdapAPI 1.0 out, as it won't be a show-stopper for ADS and Studio. It
>> doesn't matter if we continue with a shared-0.9.20 in ADS 2.0 or in Studio
>> 2.0, because ADS will be used way more often as a standalone server, or even
>> if it's embedded, the exposed interface will be the core-API part, which is
>> not part of LdapAPI (except a few classes which will need some careful
>> review).
> I wanted to save this for a separate email but just as a hint. The surface
> area of the server's API includes shared libraries, since the LDAP access
> and operation model are defined within shared modules. So there's no clear
> API boundary to the core, it slurps in shared and it becomes one big API. So
> trying to lock in core-api is futile if you don't lock in shared libraries
> to follow the same change conventions.
If we thik about subentries, for instance, it's not an issue if LdapAPI 
(which should contain those classes) is not locked down when ADS 2.0 is 
out of the door, simply because a Server user won't see those classes. 
It's for internal use only, and there is no reason for a user to 
manipulate them.

But anyway, we have to do a map of what is used and what is not, and we 
will have a better view.

Emmanuel L├ęcharny

View raw message