directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lecharny <>
Subject Partitions Heads up : resent with correct formating for the hierarchies
Date Wed, 20 Jul 2011 17:12:49 GMT
Hi guys,

sorry for having been silent for a few days, I was playing locally with 
Partitions. Let me give you some feedback on what I'm currently doing...

The idea is to refactor the way we work with Partitions. It doe snot 
make too much sense to have Partitions on one side and Store on another 
one, so I have merged those two guys (it was separated last year, but 
Alex told me that was a mistake).

The current Partition and Store hierarchies are ([] = Abstract classes, 
() = Interfaces, <> = concrete classes ) :




Once we realize that the (Avl/jdbm)Partition are backed by an 
(Avl/jdbm)Store, there is no reason not to have the classes merged. For 
convenience, we can keep the Store interface (as it exposes some methods 
specific to the backend.

Now, we can go a bit further : the Ldif partitions are all encapsulating 
an AvlPartition, because we load them in memory and it allows the system 
to create a cursor on top of them. There is no real reason not to have 
the Ldif partitions class inherit directly from the AvlPartition. That 
would give the following hierarchy :



As we keep the Store interface, we will make the 
AbstractBTreePartitionimplement it :


The SchemaPartition and the DefaultNexusPartition are two different beasts :
- the SchemaPartition have a wrapped partition, which can be either a 
Ldif partition, or a Jdbm partition, or whatever partition we want.
- the DefaultNexusPartition is just a proxy which reroute all the 
requests to the associated partition, depending on the operation DN.

Right now, this is what I'm having, and after a lot of cleanup, I have 
something which is quite consistent. It allows us to easily add a new 
Partition implementation §for instance, the HBase partition)

Now, there are some slight issues that needed to be addressed :
- first, all those Partitions now take a SchemaManager as a constructor 
parameter. This is not a problem, as the SchemaManager instance is 
always created *before* we create any partition.
- second, partitions initialization was a nightmare. We had a 
Store.init(SchemaManager) method, a final AbstractPartition.initialize() 
method and an abstract AbstractPartition.doInit() method. I removed the 
Store.init(SchemaManager), and refactored the doInti() methods in ordr 
to have a proper initialization sequence (it's not perfect though).

I do think we should merge the initialization() and doInit() method at 
some point, I don't exactly know what is the best options atm.

What remains to be done is to cleanup the DirectoryService 
initialization and the DirectoryServiceFactory initialization. 
Basically, partitions initialization is depending on a complex process, 
requiring many steps to be done :
- setting the Id
- setting the SuffixDn
- creating the underlying storage if needed
- associating an optimizer, a cursorBuilder, an evaluatorBuilder and a 
SearchEngine to the partition (for BTreePartitions)
- defining and initializing the indexes,creating the master table (for 
the BTreePartitions)
- creating the contextEntry (for the LdifPartitions)

Most of those steps are done in the doInit() methods, but it requires 
the presence of configuration elements (Id, Indexes, SuffixDn, etc) 
which *have* to be injected *before* we initialize the partition.

The way we initialize the DirectoryService requires a lot of cleanup 
too. For instance, the DefaultNexusPartition initialize the System 
partition, and all the other partitions, which does not make a lot of 
sense to me. Partitions should be associated with the DirectoryService, 
not with the DefaultNexusPartition. For the same reason, the optimizer 
and searchEngine should not be associated with a specific partition, as 
it makes thing way more complex when we try to do search across partitions.

One more thing would be to use a unique identifier across all the 
partitions. Currently, we use a Long, starting at 1L for each 
partition's contextEntry. Alex suggested that we use the entryUUID 
instead, something we can do easily, as the classes are all generic (it 
uses ID).

That's pretty much it. I'm working on my computer, because I was offline 
last week-end, but I will create a branch and commit all my changes into 
it so that you can land an eye.

Feel free to provide any feedback !

Emmanuel Lécharny

View raw message