We might have gotten lost in all the details since this is complex stuff but my point is simply not to start passing registries everywhere. I recommend to use the second form which stuffs the registries into the Serializer fed into JDBM instead of adding a new argument.  Incidentally you can pass in Serializers into JdbmTable constructors so this works out very nicely.

The registries are an implementation detail that we might want to keep hidden at this level.


On Feb 2, 2008 3:38 AM, Emmanuel Lecharny <elecharny@gmail.com> wrote:
First, I want to correct some assertion I have made : the current
Attributes serialization into the Master table is totally free of any
Java Class information. We just store the very minimum information into
the table.

More inline...

Alex Karasulu wrote:
> Hi Emmanuel,
> On Feb 1, 2008 6:31 PM, Emmanuel Lecharny <elecharny@gmail.com
> <mailto:elecharny@gmail.com>> wrote:
> ...
>     This AttributesSerializer can be rewritten and moved to handle
>     ServerEntry, assuming that :
>     1) we pass a Registries reference to the JdbmMasterTable constructor
>     2) the AttributesSerializer class is moved to core-entry
>     3) we pass the registries to this class :
>        public JdbmMasterTable( Registries registries, RecordManager recMan
>     ) throws NamingException
>        {
>            super( DBF, recMan, LONG_COMPARATOR, LongSerializer.INSTANCE,
>     new AttributesSerializer( registries ) );
>        ...
> <NOTE>
>     I have re-factored this class heavily in my private branch and it
> has been generified btw.  This is kind of cool because it means we can
> reuse this for storing and indexing any kind of objects, not just
> entries.  Anyways more points below about all this.  But please note
> that all changes here I'd like to do in the private branch so we don't
> have a serious conflict nightmare.
> </NOTE>
> What do you think about this alternative idea?  We extend a JDBM
> Serializer which can serialize and deserialize.
The Jdbm serializer is just an interface. The AttributesSerializer class
implements this interface, and it's passed to the  jdbm BTree class. The
BTree has no idea how to serialize or deserialize data, it delegates the
task to our serializer utility class. The problem is that to be able to
deserialize an Object, we *have* to have a reference to the registries,
like it or not... (otherwise, we will have to refactor the Entry
hierarchy, something I would like to avoid ...)
> Take a look at it, it's part of the JDBM API.  The subtype can be
> ServerEntrySerializer and we implement the serialize and deserialize
> methods.  Whatever creates this ServerEntrySerializer can stuff it
> with the registries so we don't have to have Registries exposed to
> these classes or to this package.
In fact, this is exactly what I proposed to do. The JdbmMaterTable is
our, so we can either pass the registries as I did (look ate the code I
posted), _or_, and this may be what you have in mind, instead of passing
the registries, we pass a reference to the serializer, like in :

   public JdbmMasterTable( Serializer entrySerializer, RecordManager
recMan ) throws NamingException
       super( DBF, recMan, LONG_COMPARATOR, LongSerializer.INSTANCE,
entrySerializer );

The impact is very minimal here : in the BootsrapPlugin class you invoke
the JdbmStore initialization with this serializer instanciation :

   private void initializePartition( File workingDirectory ) throws
MojoFailureException, NamingException
           store.init( new ServerEntrySerializer( registries ) ); //
Instead of a store.init() call

then in the JdbmStore, you propagate this serializer instance :

   public synchronized void init( Registries registries, Serializer
entrySerializer )
           throws NamingException
       // Create the master table (the table wcontaining all the entries)
       master = new JdbmMasterTable( entrySerializer, recMan );

and we are done. Compared to what I have done, it's quite equivalent.
The only difference is that I passed the registries, and initialized the
serializer down in the JdbmTable. here, we pass the serializer instance
instead. Anyway, we will have to change the JdbmTable interface.

Last point : instead of passing an OidRegistry and AttributeTypeRegistry
references to the JdbmStore.init() method, I suggest we pass the
registries. One less parameter ;)

> Another thing we could do is create our own JDBM neutral/independent
> interface with these serialize and deserialize methods: call it
> Marshaller.
Very true ! We have to be totally decoupled from Jdbm Serialize
Interface, as if we want to use another backend, we can't be stuck with
a specific API.
> We can create an implementation that specifically handles the
> [de]serialization of ServerEntry objects.  And we can make this
> parameterized (generified).  Then we can mandate that the
> MasterTable<E> class take a Marshaller<E> in it's constructor.  The
> constructor can then wrap this up in a Serializer inner class
> specifically suited for JDBM.  We can make sure the reference to the
> Marshaller is a transient handle that can be set when we create a
> MasterTable in case JDBM does something funny and desides to store
> this thing into it's db files.
Yes, definitively. I thought we need to do that at some point, but it
was not my main concern... We need it for after 2.0, but we can
implement it for 2.0, that's for sure !
> Now this Marshaller can be used anywhere we need to serialize the
> entry, over the network with Mitosis or to disk with the JDBM and
> potentially other partition implementations.
This is a very valid point.

I have also another concern about the current serialize : the current
serializer transform an object to a byte[]. This is _bad_ ! Just imagine
what happens when you serialize JPegPhoto... We need to wrap a
StreamBuffer, like what Java does to serialize objects.
>     So, wdyt ? Is it OK to modify the API this way ?
> I think the proposed mechanism will create too much interdependency
> especially between modules.
well, you have to modify the API anyway ...

cordialement, regards,
Emmanuel Lécharny