Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: neutral (asf.osuosl.org: local policy)
Subject: Re: [apacheds] Idea on refactoring Database, ContextPartition, and
	RootNexus
From: Emmanuel Lecharny <elecharny@apache.org>
To: Apache Directory Developers List <dev@directory.apache.org>
In-Reply-To: <768dcb2e05061902262c150551@mail.gmail.com>
References: <768dcb2e05061821154e2aa35d@mail.gmail.com>
	 <1119170532.7400.39.camel@wkslx01.iktek.com>
	 <768dcb2e05061902262c150551@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Organization: iktek
Date: Sun, 19 Jun 2005 12:01:08 +0200
Message-Id: <1119175268.7400.79.camel@wkslx01.iktek.com>
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi Trustin,

> If you're going to change an entry, put operation will be sufficient. 
> But if underlying database implementaion stores attributes sparsely,
> it can be insuffucient of course.  I guess that kind of implementation
> will be rare and can be improved by using cache IMHO, so the problem
> here is possibly moving or renaming.

Not so sure. You have two basic choices here :
 - use a specific indexed Database (BDB, or whatever fit your needs)
 - use a classic relationnal Database (Or*cle(TM), Posgresql,
MySql, ...)

I bet that if you are to use ApacheDS in production, you will favor
reliability above performance. This will be the choice of many clients,
I assure you ! Not that performance is not important, but clients hate
to loose data, strange enough ;)

Or*cle(TM) has implemented a new hierarchical engine in V10, which
improve greatly performances for data which has this kind of structure. 

Whatever, I really buy your idea to have the simplest interface as
possible, because it will help to implements any kind of backend
Databse. 

There is also a point that I want to rise : a Modify request does not
give you the attributes of an entry, as you perfectly know. So a Put
will be quite complicated to implement, as you won't be able to deal
with deleted attributes without fetching the full entry before. So the
sequence will be :
 - fetch the entry from the Database
 - modify the entry in memory
 - update the entry into the database

It will be very costly. 

I also bet that we won't have ten thousands of Database to deal with, so
even if the proposed interface is a little bit more complicated than
just a Put and a Delete, it's not a big deal.

> 
> The reason why I proposed these simple operations were actually to
> create an interceptor that supresses delete operation and marks the
> target entry as 'deleted' instead of deleting it really.  

I guess that you'll use it for Mitosis?

> I was able
> to simplify the implementation of that filter using this ways.  But
> now I found that we can implement NotDeletingDatabase which wraps
> existing Database to intercept delete, move or modifyRN operations and
> fulfill my aim, and of course, retaining current Database interface.

So your NotDeletingDatabase is still seen as a Database, am I wrong? If
so, this is quite a smart idea. The end user should not know what's
going on in the kitchen, he just want to have good food in his plate !


> So.. here's my more refined idea:
> 
> 1) create a new package 'partition' and place all partition related
> stuffs there for better package layout.
> 2) merge AbstractContextPartition and ApplicationContextPartition to
> DatabaseContextPartition.
> 3) move db package into 'partition' package. (i.e. partition.database)
> and put DatabaseContextPartition to 'database' package for better
> package layout.
> 4) retain the interface of current Database class, and add transaction
> support to make all operations atomic.

Much more important : just add transaction because it's a must. This is
something I always had problem with OpenLdap, because updates are
dangerous on a running environment. If you change an information, you
can't guaranty that somebody is already using it. The problem is that
you have to extend the transaction mechanism to reads, because you want
a reader to have a fresh entry, not something that is currently being
modified. (Updating en entry can be something costly, if you store
images, so this is a scenario which could occur)

> 5) add AbstractDatabase class that helps users to implement databases
> easily.  (e.g. two modify operations are delegated to one modify
> method after some normalization)

+1, but can you develop a little bit what you mean by 'two modify
operations are delegated to one modify method after some
normalization' ?

> 6) provide standardized initialization method like 'open' instead of
> constructors like we did for ContextPartition so that users can
> instantiate Database and DatabaseContextPartition in configuration
> phase.
+1

> 
> I must admit that my first idea was a kind of expression of
> stupidness.  

-1. You are not stupid at all !

> > Put aside the previous comment, I really like the Tx stuff. This is
> > something that many LDAP implementations don't have.
> 
> Yeah we'll be able to put quite useful metadata to Database. Cool? :)

really cool, and we need it actually, we will have 30° today here in
Paris ;)

Emmanuel Lécharny