Mailing-List: contact ojb-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "OJB Developers List" <ojb-dev@db.apache.org>
Message-ID: <6FB083FB72EFD21181D30004AC4CA18A03249ACD@srv002>
From: Charles Anthony <charles.anthony@hpdsoftware.com>
To: 'OJB Developers List' <ojb-dev@db.apache.org>
Subject: RE: Foreign Key Conflicts in ODMG
Date: Mon, 6 Oct 2003 16:53:39 +0100 
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"

Hi Thomas et Al,
[...]
> 
> > The question is, why is this so ? It "feels" a bit like a 
> bubble-sort not
> > being done enough times e.g. perhaps we should re-order 
> until there aren't
> > any more changes to do... 
> 
> By having a short look at the code I think that you could be 
> right. I'm 
> not sure that the current algorithm will work correctly to for an 
> arbitrary depth of recursive dependencies.
> 
> > 
> > Why does reorder build a new hashtable of object envelopes ?
> 
> AFAICR this was done to avoid ConcurrentModificationExceptions.
> 
> > ps. I don't think this is an rc4 specific problem. 
> 
> I agree. This issue need some further investigation. 
> Unfortunately I'm 
> quite busy with a high priority project and don't have much time left 
> for OJB :-(


I've modified ObjectEnvelopeTable to repeatedly reorder until there are no
more changes to be made. This solves my immediate problem, with no negative
effects. However, I can see that it might cause a problem if you have a
mutual dependency (a bi-directional 1-1 relationship) - we'd never stop
sorting. I suspect that - in the long term - the ordering algorithm needs to
be looked at by someone who is smarter than me !

Anyway, my (minimally changed) source is included inline; use it however you
see fit.

Cheers,

Charles.


package org.apache.ojb.odmg;

/* ====================================================================
 * The Apache Software License, Version 1.1
 *
 * Copyright (c) 2001 The Apache Software Foundation.  All rights
 * reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. The end-user documentation included with the redistribution,
 *    if any, must include the following acknowledgment:
 *       "This product includes software developed by the
 *        Apache Software Foundation (http://www.apache.org/)."
 *    Alternately, this acknowledgment may appear in the software itself,
 *    if and wherever such third-party acknowledgments normally appear.
 *
 * 4. The names "Apache" and "Apache Software Foundation" and
 *    "Apache ObjectRelationalBridge" must not be used to endorse or promote
products
 *    derived from this software without prior written permission. For
 *    written permission, please contact apache@apache.org.
 *
 * 5. Products derived from this software may not be called "Apache",
 *    "Apache ObjectRelationalBridge", nor may "Apache" appear in their
name, without
 *    prior written permission of the Apache Software Foundation.
 *
 * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
 * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
 * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 * ====================================================================
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of the Apache Software Foundation.  For more
 * information on the Apache Software Foundation, please see
 * <http://www.apache.org/>.
 */

import java.util.ArrayList;
import java.util.Collection;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import org.apache.commons.lang.builder.ToStringBuilder;
import org.apache.commons.lang.builder.ToStringStyle;
import org.apache.ojb.broker.Identity;
import org.apache.ojb.broker.ManageableCollection;
import org.apache.ojb.broker.OJBRuntimeException;
import org.apache.ojb.broker.OptimisticLockException;
import org.apache.ojb.broker.PersistenceBroker;
import org.apache.ojb.broker.PersistenceBrokerFactory;
import org.apache.ojb.broker.accesslayer.ConnectionManagerIF;
import org.apache.ojb.broker.metadata.ClassDescriptor;
import org.apache.ojb.broker.metadata.CollectionDescriptor;
import org.apache.ojb.broker.metadata.ObjectReferenceDescriptor;
import org.apache.ojb.broker.util.ArrayIterator;
import org.apache.ojb.broker.util.logging.Logger;
import org.apache.ojb.broker.util.logging.LoggerFactory;
import org.apache.ojb.odmg.locking.LockManagerFactory;
import org.apache.ojb.odmg.states.StateOldClean;
import org.odmg.LockNotGrantedException;
import org.odmg.Transaction;
import org.odmg.TransactionAbortedException;

/**
 * manages all ObjectEnvelopes included by a transaction.
 * Performs commit, and rollack operations on all included Envelopes.
 *
 * @author Thomas Mahler
 * @author <a href="mailto:mattbaird@yahoo.com">Matthew Baird</a>
 *
 * MBAIRD: added explicit closing and de-referencing to prevent any
 * GC issues.
 */

public class ObjectEnvelopeTable
{
    private Logger log = LoggerFactory.getLogger(ObjectEnvelopeTable.class);
    private TransactionImpl transaction;

    /**
     * the internal table mapping Objects to their ObjectTransactionWrappers
     */
    private Map mhtObjectEnvelopes = new HashMap();

    /**
     * a vector containing the ObjectEnvelope objects representing
modifications
     * in the order they were added. If an ObjectEnvelope is added twice,
only
     * the the second addition is ignored.
     */
    private ArrayList mvOrderOfIds = new ArrayList();

    /**
     * marker used to avoid superfluous reordering and commiting
     */
    private boolean needsCommit = false;

    /**
     * prepare this instance for reuse
     */
    public void refresh()
    {
        /**
         * MBAIRD: be nice and remove all references so they can be
         * gc'd
         */
        if (mhtObjectEnvelopes != null)
        {
            Iterator iter = mvOrderOfIds.iterator();
            ObjectEnvelope temp;
            while (iter.hasNext())
            {
                temp = (ObjectEnvelope) mhtObjectEnvelopes.get(iter.next());
                temp.close();
            }
        }
        needsCommit = false;
        mhtObjectEnvelopes.clear();
        mvOrderOfIds.clear();
    }

    /**
     * Creates new ObjectEnvelopeTable
     */
    public ObjectEnvelopeTable(TransactionImpl myTransaction)
    {
        transaction = myTransaction;
    }

    /**
     * perform commit on all tx-states
     */
    public void commit() throws TransactionAbortedException,
LockNotGrantedException
    {
        PersistenceBroker broker = transaction.getBroker();
        ConnectionManagerIF connMan = broker.serviceConnectionManager();
        boolean saveBatchMode = connMan.isBatchMode();

        try
        {
            if (log.isDebugEnabled())
            {
                log.debug(
                    "PB is in internal tx: "
                        + broker.isInTransaction()
                        + "  broker was: "
                        + broker);
            }
            // all neccessary db operations are executed within a
PersistenceBroker transaction:
            if (!broker.isInTransaction())
            {
//                if (log.isDebugEnabled())
//                    log.debug("call beginTransaction() on PB instance");
//                broker.beginTransaction();
                log.error("PB associated with current odmg-tx is not in
tx");
                throw new TransactionAbortedException("Underlying PB is not
in tx");
            }

            // Committing has to be done in two phases. First implicitly
upgrade to lock on all related
            // objects of objects in this transaction. Then the list of
locked objects has to be
            // reordered to solve referential integrity dependencies, then
the objects are
            // written into the database.

            // 0. turn on the batch mode
            connMan.setBatchMode(true);

            // 1. upgrade implicit locks.
            upgradeImplicitLocksAndCheckIfCommitIsNeeded();

            // 2. Repeatedly reorder objects until no changes occur
            boolean changeOccured = false;
            do {
              changeOccured = reorder();
            } while(changeOccured);

            // 3. commit objects.
            commitAllEnvelopes(broker);

            // 4. execute batch
            connMan.executeBatch();

            // 5.Update all Envelopes to new CleanState
            setCleanState();

        }
        catch (Throwable t)
        {
            // we do that in TransactionImpl#abort()
            //            broker.abortTransaction();
            connMan.clearBatch();
            log.error("Commit on object level failed for tx " + transaction,
t);
            if (t instanceof OptimisticLockException)
            {
	            // PB OptimisticLockException should be clearly
signalled to the user
            	throw new LockNotGrantedException(t.getMessage());
            }
			      else
            {
                    throw new TransactionAbortedExceptionOJB(t);
            }
        }
        finally
        {
            needsCommit = false;
            connMan.setBatchMode(saveBatchMode);
        }
    }

    /**
     * commit all envelopes against the current broker
     * @param broker the PB to persist all objects
     */
    private void commitAllEnvelopes(PersistenceBroker broker)
    {
        if (needsCommit)
        {
            Iterator iter;
            // using clone to avoid ConcurentModificationException
            iter = ((List) mvOrderOfIds.clone()).iterator();
            while (iter.hasNext())
            {
                ObjectEnvelope mod = (ObjectEnvelope)
mhtObjectEnvelopes.get(iter.next());
                mod.getModificationState().commit(mod, broker);
            }
        }
    }

	/**
	 * commit all envelopes against the current broker
	 * @param broker the PB to persist all objects
	 */
	private void setCleanState()
	{
		 if (needsCommit)
		 {
			  Iterator iter;
			  // using clone to avoid
ConcurentModificationException
			  iter = ((List) mvOrderOfIds.clone()).iterator();
			  while (iter.hasNext())
			  {
					ObjectEnvelope mod =
(ObjectEnvelope) mhtObjectEnvelopes.get(iter.next());
					if(mod.getModificationState() !=
StateOldClean.getInstance())
					{
						mod.manage(mod.getObject());
	
mod.setModificationState(StateOldClean.getInstance());
					}
			  }
		 }
	}

    /**
     * Implicitely upgrade locks on modified objects.
     * Also checks if there are any operations to commit.
     */
    private void upgradeImplicitLocksAndCheckIfCommitIsNeeded()
    {
        boolean useImplicitLocking =
getConfiguration().useImplicitLocking();
        // using clone to avoid ConcurentModificationException
        Iterator iter = ((List) mvOrderOfIds.clone()).iterator();
        while (iter.hasNext())
        {
			boolean markDirty = false;
            ObjectEnvelope mod = (ObjectEnvelope)
mhtObjectEnvelopes.get(iter.next());
            if (log.isDebugEnabled())
                log.debug("commit: " + mod);
            // if the Object has been modified by transaction, mark object
as dirty
            // but only if it has not been marked during tx already !!
            if ((!mod.needsDelete()) && (!mod.needsInsert()) &&
(!mod.needsUpdate()))
            {
				/**
				 * second check is, has the object in the
envelope changed.
				 */
                if (mod.hasChanged())
                {
					/**
					 * now, the quickest thing to check
is the useImplicitLocking flag. If we are using
					 * implicit locking, let's try to
upgrade the lock, and mark the markDirty
					 */
					if (useImplicitLocking)
					{
						// implicitely acquire a
write lock !
	
transaction.lock(mod.getObject(), Transaction.UPGRADE);
						// objects needs commit
action, thus set markDirty to true:
						markDirty = true;
					}
					/**
					 * If useImplicitLocking is false,
we still need to check if the object in the envelope
					 * is write locked. If it is, we
don't have to upgrade the lock, just mark markDirty
					 */
					else if
(LockManagerFactory.getLockManager().checkWrite(transaction,
mod.getObject()))
					{
						// objects needs commit
action, thus set markDirty to true:
						markDirty = true;
					}
					if (markDirty)
					{
						needsCommit=true;
						// mark object dirty
	
mod.setModificationState(mod.getModificationState().markDirty());
					}
                }
            }
            else
            {
                // objects needs commit action, thus set needCommit to true:
                needsCommit = true;
            }
        }
    }

    /**
     * perform rollback on all tx-states
     */
    public void rollback()
    {
        try
        {
            PersistenceBroker broker = transaction.getBroker();
            Iterator iter = mvOrderOfIds.iterator();
            while (iter.hasNext())
            {
                ObjectEnvelope mod = (ObjectEnvelope)
mhtObjectEnvelopes.get(iter.next());
                if(mod!=null){
                if (log.isDebugEnabled())
                    log.debug("rollback: " + mod);
                // if the Object has been modified has been modified by
transaction, mark object as dirty
                if (mod.hasChanged())
                {
 
mod.setModificationState(mod.getModificationState().markDirty());
                }
                mod.getModificationState().rollback(mod, broker);
                }
            }
        }
        finally
        {
            needsCommit = false;
        }
    }

    /**
     * remove an objects entry from the Hashtable
     */
    public void remove(Object pKey)
    {
        Identity id = null;
        if (pKey instanceof Identity)
        {
            id = (Identity) pKey;
        }
        else
        {
            id = new Identity(pKey, transaction.getBroker());
        }
        mhtObjectEnvelopes.remove(id);
        mvOrderOfIds.remove(id);
    }

    /**
     *
     * Get an enumeration of all the elements in this ObjectEnvelopeTable
     * in random order.
     *
     * Creation date: (11.02.2001 12:45:08)
     *
     * @return Enumeration an enumeration of all elements managed by this
ObjectEnvelopeTable
     *
     */
    public Enumeration elements()
    {
        return
java.util.Collections.enumeration(mhtObjectEnvelopes.values());
    }

    /**
     * retrieve an objects ObjectModification state from the hashtable
     */
    public ObjectEnvelope getByIdentity(Identity id)
    {
        return (ObjectEnvelope) mhtObjectEnvelopes.get(id);
    }

    /**
     * retrieve an objects ObjectEnvelope state from the hashtable.
     * If no ObjectEnvelope is found, a new one is created and returned.
     * @return the resulting ObjectEnvelope
     */
    public ObjectEnvelope get(Object pKey)
    {
        Identity id = new Identity(pKey, transaction.getBroker());
        //Integer keyInteger = new Integer(System.identityHashCode(key));
        ObjectEnvelope result = (ObjectEnvelope) mhtObjectEnvelopes.get(id);
        if (result == null)
        {
            result = new ObjectEnvelope(pKey, transaction);
            mhtObjectEnvelopes.put(id, result);
            mvOrderOfIds.add(id);
            if (log.isDebugEnabled())
                log.debug("register: " + result);
        }
        return result;
    }

    /**
     * store an objects transactional state into the Hashtable
     */
    public void put(Object pKey, ObjectEnvelope modification)
    {
        Identity id = new Identity(pKey, transaction.getBroker());
        //Integer keyInt = new Integer(System.identityHashCode(key));
        if (log.isDebugEnabled())
            log.debug("register: " + modification);
        if (!mhtObjectEnvelopes.containsKey(id))
            mvOrderOfIds.add(id);
        mhtObjectEnvelopes.put(id, modification);
    }

    /**
     * Returns a String representation of this object
     */
    public String toString()
    {
        ToStringBuilder buf = new ToStringBuilder(this,
ToStringStyle.MULTI_LINE_STYLE);
        buf.append("### ObjectEnvelopeTable dump:");
        Enumeration enum = elements();
        while (enum.hasMoreElements())
        {
            ObjectEnvelope mod = (ObjectEnvelope) enum.nextElement();
            buf.append(mod.toString());
        }
        return buf.toString();
    }

    /**
     * retrieve an objects ObjectModification state from the hashtable
     */
    public boolean contains(Object pKey)
    {
        Identity id = new Identity(pKey, transaction.getBroker());
        //Integer keyInteger = new Integer(System.identityHashCode(key));
        return mhtObjectEnvelopes.containsKey(id);
    }

    /**
     * Reorder the objects in the table to resolve referential integrity
dependencies.
     * @return true if anything changed places, false if not.
     */
    private boolean reorder() throws IllegalAccessException
    {
      boolean orderChanged  = false;

        if (needsCommit)
        {
            ArrayList vNewVector = new ArrayList(mvOrderOfIds.size());
            Map htNewHashtable = new HashMap((int) (mvOrderOfIds.size() *
1.1), 1f);
            Map htOldVectorPosition = new HashMap((int) (mvOrderOfIds.size()
* 1.1), 1f);
            for (int i = 0; i < mvOrderOfIds.size(); i++)
                htOldVectorPosition.put(mvOrderOfIds.get(i), new
Integer(i));
            for (int i = 0; i < mvOrderOfIds.size(); i++)
            {
                Identity id = (Identity) mvOrderOfIds.get(i);
                if (id != null)
                {
                    mvOrderOfIds.set(i, null);

                    ObjectEnvelope o = (ObjectEnvelope)
mhtObjectEnvelopes.get(id);
                    mhtObjectEnvelopes.remove(id);
                    reorderObject(htNewHashtable, vNewVector, o, id,
htOldVectorPosition);
                }
            }

            /* Let's find out if anything changed places */
            for(int i=0; i< vNewVector.size() && !orderChanged; i++){
              Identity id = (Identity) vNewVector.get(i);
              Integer oldPosition = (Integer) htOldVectorPosition.get(id);

              if(oldPosition.intValue()!=i){
                orderChanged=true;
              }
            }

            mvOrderOfIds = vNewVector;
            mhtObjectEnvelopes = htNewHashtable;
        }
      return orderChanged;
    }

  /**
     * put an object and all its dependent objects in the new vector. If the
object
     * in question is going to be DELETEd, first the objects referenced in
collections
     * are put in the vector, then the object in question and then the
references.
     * Otherwise the order is reversed.
     */
    private void reorderObject(
        Map htNewHashtable,
        List newVector,
        ObjectEnvelope objectToReorder,
        Identity id,
        Map htOldVectorPosition)
        throws IllegalAccessException
    {
        PersistenceBroker broker = transaction.getBroker();
        if (objectToReorder != null)
        {
            ClassDescriptor cld =
broker.getClassDescriptor(objectToReorder.getObject().getClass());
            if (objectToReorder.needsDelete())
            {
                reorderCollection(
                    htNewHashtable,
                    newVector,
                    objectToReorder,
                    cld,
                    htOldVectorPosition);
                newVector.add(id);
                htNewHashtable.put(id, objectToReorder);
                reorderReference(
                    htNewHashtable,
                    newVector,
                    objectToReorder,
                    cld,
                    htOldVectorPosition);
            }
            else
            {
                reorderReference(
                    htNewHashtable,
                    newVector,
                    objectToReorder,
                    cld,
                    htOldVectorPosition);
                newVector.add(id);
                htNewHashtable.put(id, objectToReorder);
                reorderCollection(
                    htNewHashtable,
                    newVector,
                    objectToReorder,
                    cld,
                    htOldVectorPosition);
            }
        }
    }

    /**
     * Resolves all objects referenced in collections of objectToReorder.
The referenced
     * objects are removed from the list of modified objects in this class
and passed
     * to reorderObject, where dependencies of the object are resolved (i.e.
if the referenced
     * object contains again collections or references) and it is finally
put into the new order
     * vector.
     */
    private void reorderCollection(
        Map htNewHashtable,
        List newVector,
        ObjectEnvelope objectToReorder,
        ClassDescriptor cld,
        Map htOldVectorPosition)
        throws IllegalAccessException
    {
        Iterator i;
        i = cld.getCollectionDescriptors().iterator();
        while (i.hasNext())
        {
            CollectionDescriptor cds = (CollectionDescriptor) i.next();
            Object col =
cds.getPersistentField().get(objectToReorder.getObject());
            if (col != null)
            {
                Iterator colIterator;
                if (col instanceof ManageableCollection)
                {
                    colIterator = ((ManageableCollection)
col).ojbIterator();
                }
                else if (col instanceof Collection)
                {
                    colIterator = ((Collection) col).iterator();
                }
                else if (col.getClass().isArray())
                {
                    colIterator = new ArrayIterator(col);
                }
                else
                {
                    throw new OJBRuntimeException(
                        col.getClass()
                            + " can not be managed by OJB, use Array,
Collection or ManageableCollection instead !");
                }
                while (colIterator.hasNext())
                {
                    // The collection now contains all the objects in the
collection.
                    // Now we have to retrieve the ObjectEnvelope
representing this
                    // Object from the hashtable, remove it from the vector
and reorder
                    // the retrieved ObjectEnvelope.
                    Identity id = new Identity(colIterator.next(),
transaction.getBroker());
                    ObjectEnvelope oe = (ObjectEnvelope)
mhtObjectEnvelopes.get(id);
                    if (oe != null)
                    {
                        mvOrderOfIds.set(((Integer)
htOldVectorPosition.get(id)).intValue(), null);
                        // mvOrderOfIds.set(mvOrderOfIds.indexOf(id), null);
                        mhtObjectEnvelopes.remove(id);
                        reorderObject(htNewHashtable, newVector, oe, id,
htOldVectorPosition);
                    }
                }
            }
        }
    }

    /**
     * Resolves all objects referenced in references of objectToReorder. The
referenced
     * objects are removed from the list of modified objects in this class
and passed
     * to reorderObject, where dependencies of the object are resolved (i.e.
if the referenced
     * object contains again collections or references) and it is finally
put into the new order
     * vector.
     */
    private void reorderReference(
        Map htNewHashtable,
        List newVector,
        ObjectEnvelope objectToReorder,
        ClassDescriptor cld,
        Map htOldVectorPosition)
        throws IllegalAccessException
    {
        Iterator i = cld.getObjectReferenceDescriptors().iterator();
        while (i.hasNext())
        {
            ObjectReferenceDescriptor rds = (ObjectReferenceDescriptor)
i.next();
            Object refObj =
rds.getPersistentField().get(objectToReorder.getObject());
            if (refObj != null)
            {
                Identity id = new Identity(refObj, transaction.getBroker());
                ObjectEnvelope oe = (ObjectEnvelope)
mhtObjectEnvelopes.get(id);
                if (oe != null)
                {
                    mhtObjectEnvelopes.remove(id);
                    mvOrderOfIds.set(((Integer)
htOldVectorPosition.get(id)).intValue(), null);
                    // mvOrderOfIds.set(mvOrderOfIds.indexOf(id), null);
                    reorderObject(htNewHashtable, newVector, oe, id,
htOldVectorPosition);
                }
            }
        }
    }

    /**
     * get Configuration
     * @return OdmgConfiguration
     */
    private OdmgConfiguration getConfiguration()
    {
        OdmgConfiguration config =
            (OdmgConfiguration)
PersistenceBrokerFactory.getConfigurator().getConfigurationFor(
                null);
        return config;
    }

} 


This email and any attachments are strictly confidential and are intended
solely for the addressee. If you are not the intended recipient you must
not disclose, forward, copy or take any action in reliance on this message
or its attachments. If you have received this email in error please notify
the sender as soon as possible and delete it from your computer systems.
Any views or opinions presented are solely those of the author and do not
necessarily reflect those of HPD Software Limited or its affiliates.

 At present the integrity of email across the internet cannot be guaranteed
and messages sent via this medium are potentially at risk.  All liability
is excluded to the extent permitted by law for any claims arising as a re-
sult of the use of this medium to transmit information by or to 
HPD Software Limited or its affiliates.


---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org