Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Message-ID: <4B0B2156.3050205@apache.org>
Date: Tue, 24 Nov 2009 00:57:10 +0100
From: =?ISO-8859-1?Q?Emmanuel_L=E9charny?= <elecharny@apache.org>
Reply-To: elecharny@apache.org
Organization: The Apache Software Foundation
User-Agent: Thunderbird 2.0.0.23 (X11/20090817)
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: [Schema Refactoring] Serious side effect...
References: <4B0A4866.4090101@nextury.com>
 <a32f6b020911231105o230a1a4bx2e116589f9b9c72e@mail.gmail.com>
 <4B0B1890.9090709@nextury.com> <4B0B1B5F.1000100@symas.com>
In-Reply-To: <4B0B1B5F.1000100@symas.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Howard Chu wrote:
> Emmanuel Lecharny wrote:
>> <snip/>
>> Any other operation (delete, modify, rename, disabling a schema) are 
>> most certainly leading to dire errors, something an administrator 
>> will not want to experiment in production. IMO, they should be 
>> forbidden on a working base. Such operation is like manipulating a 
>> loaded weapon with no safety...
>
> This is pretty much the same conclusion we reached, which is why in 
> OpenLDAP 2.3 we only supported dynamic adding of schema. In 2.4 we 
> support delete/modify but it's a hack - the deleted elements are kept 
> around. If you do a Modify to alter an existing value (e.g. 
> Modify/delete foo=1/add foo=2) we make sure the subsequent add applies 
> to the corresponding deleted element. Since we don't refcount 
> AttributeDescriptions, these things are kept around for the life of 
> the server process, and only get purged on a restart.
Funny enough, this was what I got in ADS, until the point I decided that 
I should clean the old schema, to avoid memory leaks. Too bad !!! All 
the objects pointing to the deleted schema were doomed ! That led to 
this thread...
>
>> For the reason I mentionned, I don't think that any alternative is ok.
>> We probably don't have a perfect solution because there are none. As we
>> say : "any problem vanishes when there is no solution"...
>>
>> More seriously, I don't think we need a dynamic schemaManager for a LDAP
>> server in production : admins don't change such a critical thing in
>> production, except those who are insane or desesperate. We must accept
>> the idea that we might have a downtime, we just have to minimize it.
>
> That was my generally my perspective as well. IMO, admins should be 
> able to dynamically Add tested schema to a production server. If 
> they're experimenting and need to alter the schema on-the-fly, they 
> should be testing in a dedicated test environment. This was the 
> rationale behind the 2.3 implementation.
Make totally sense to me. But ...
>
> But it seems that admins like to whine a lot about things they think 
> they need, even if they're more likely to shoot themselves in the 
> foot. So we added dynamic delete/modify support in 2.4.
There is also a special use case we are aware of : people using an 
embedded server want to keep it running, forever. This is a reason why 
they might want dynamic schema modifications.

Anyway, the desire to change a schema in production, embedded or not, is 
probably sick. The larger the base, the sicker this request is : it's a 
bit like trying to fix your engine while driving on the highway at 60 
mph...

The few users I know who have huge database (from 100 000 entries to 70 
00 000 entries) don't change the schema *at all*. The service don't go 
in production if the schema is wrong, anyway. It's like changing a 
database schema in production...

Let's be realistic : it can happen, but people have to pay the price for 
their under tested system. Considering that a LDAP server is mainly used 
for searches, the downtime to export the data, massage them accordingly 
to the new schema, inject them in another server, index everyting, 
launch the new server, stop the old one, inject the data updated between 
the moment the extract has been done and the moment the old server has 
been stopped, is probably very limited.
My own experience (with an OpenLDAP server - 2.4.16, and 5 millions 
entries) demonstrate that the downtime is around 5 minutes if we stop 
all the server before exporting the data and reimport them. Probably 
acceptable, if you are not managing the Kennedy Airport with this system 
! Even if it's a one hour downtime, twice a year, this is a 99.98% uptime.

FYI, Amazon EC2 SLA is 99.95% uptime, a 4.38 hours/year downtime. Here, 
what is important is not the downtime, but the failover solution, and 
more important, the fact that you *select* the downtime period. Sundays 
are not only a day off : they are a perfect day for maintenance operations !
>
> Either way, 2.3 or 2.4, it's possible to end up with entries in the DB 
> using schema that no longer exist - we don't search them out and 
> remove them when deleting schema elements. (Indeed, you really 
> shouldn't; someone may be adding a new version of the deleted schema 
> in the next operation. You have no way to know when a delete is really 
> final, and spontaneously deleting user data is always a mistake...)
Make sense. Again, Admin must *know* what they are doing. Aren't they ? 
... Hopefully, we provide expensive consulting to those guys ;)

-- 
--
cordialement, regards,
Emmanuel L�charny
www.iktek.com
directory.apache.org