db-jdo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörg von Frantzius <joerg.von.frantz...@artnology.com>
Subject Re: makePersistent detached instance deleted on database
Date Sat, 11 Mar 2006 23:42:57 GMT
Hello Craig,

Craig L Russell schrieb:
> Hi Jörg,
>
> What I'm looking for first of all is a description of what replication 
> means to you. 
>
> Is it a master/slave configuration where only one of the datastores is 
> updating instances, and you're pushing out updates? 
>
> Is bidirectional update supported?
>
> Does it support application identity as well as datastore identity? 
>
> Is versioning supported?
We're using timestamps issued by the master datastore to find changes.

Or do you mean optimistic versioning here?
> Is there  a need for user-written callbacks in case of version failure?
>
> It appears from the description of your proposed feature enhancements 
> that you're only covering master/slave, application identity, no 
> versioning, no bidirectional update.
That's exactly the kind of replication we're doing here!

Fortunately bidirectional replication wasn't a requirement, as one would 
have to either think of some kind of distributed locking scheme, or 
conflict resolution. I'm not sure whether that should make any 
difference for the JDO API, unless we'd want to provide support for 
conflict resolution e.g. by reflecting upon individual fields that have 
conflicting changes. Are you thinking of support for one or both of the 
two approaches in the spec?

Regards,
Jörg
> Craig
>
> On Mar 10, 2006, at 9:22 AM, Jörg von Frantzius wrote:
>
>> Hi Craig,
>>
>> sorry I didn't understand what you wanted to see here exactly. Maybe 
>> something like this:
>>
>> Replication does work perfectly with detaching using an attachCopy() 
>> that inserts new instances, and it does so in a productive 
>> environment using an older JPOX version, however compliant with the 
>> spec that was.
>>
>> With the current specification, there is one thinkable approach that 
>> might work (as pointed out by Matthew), and it has two drawbacks. 
>> Here's what it could look like:
>>
>>     Object detachedGraph = pm1.detachCopy(root);
>>     try {
>>         pm2.makePersistent(detachedGraph);
>>     } catch (whatever Exception it is exactly) {
>>         Collection failedInstances = ... // failed are contained in exception
>>         // find out the original instances of the failed detached instances
>>         Collection originalFailedInstances; 
>>         for (Object failed: failedInstances) {
>>             Object originalFailed  = pm1.getObjectById(JDOHelper.getObjectId(failed));
>>     	originalFailedInstances.add(originalFailed);
>>         }
>>         pm1.makeTransientAll(originalFailedInstances);
>>         pm2.makePersistentAll(originalFailedInstances);
>>         // now that the new objects are made persistent, this should succeed:
>>         pm2.makePersistent(detachedGraph);
>>     }
>>
>> The drawbacks are:
>>
>>    1. it *won't work for datastore identity*, as the identities of
>>       the instances new to the second PM/datastore are lost upon
>>       makeTransient(), and there is no reliable way to assure that
>>       they will be assigned the same identities upon makePersistent()
>>       on the second PM,
>>    2. it certainly is *not very effective*.
>>
>> The remedy could be the approach proposed by Marco: to have an 
>> additional method
>>
>>     PersistenceManager.makePersistent(Object o, boolean insertNew)
>>
>> and to have PersistenceManager.makePersistent(Object) default to 
>> PersistenceManager.makePersistent(o, false).
>>
>> Regards,
>> Jörg
>>
>> Craig L Russell schrieb:
>>> Hi Jörg,
>>>
>>> As I said earlier, I'd like to see the details of using this feature 
>>> for replication. What cases are covered, what cases are still 
>>> problematic, what modifications to the specification are needed in 
>>> order to accomplish the task?
>>>
>>> Craig
>>>
>>> On Mar 10, 2006, at 3:14 AM, Jörg von Frantzius wrote:
>>>
>>>> Hi Craig,
>>>>
>>>> replication really is lost in a specification gap: makePersistent() 
>>>> on transient instances won't update existing data, and on detached 
>>>> instances it won't insert new. For replication, you need both 
>>>> behaviours at the same time.
>>>>
>>>> That's really misfortunate for such a nice feature! Even more so as 
>>>> it is not just theory, but it proves to be working in production 
>>>> with JPOX' old implementation of attachCopy().
>>>>
>>>> Regards,
>>>> Jörg
>>>>
>>>> Craig L Russell schrieb:
>>>>> Hi Jörg,
>>>>>
>>>>> Using detachment for replication is an interesting use case, and 
>>>>> I'd like to see more in-depth analysis of the issues that you 
>>>>> encounter once you've done with it.
>>>>>
>>>>> The use-case for detachment is long-running optimistic 
>>>>> transactions, as you have noted below. We did add 
>>>>> makeTransient(Object, useFetchPlan) as a way to disconnect objects 
>>>>> from one datastore that could be used with another, but I really 
>>>>> doubt that we are going to be able to incorporate into the JDO API 
>>>>> all the policy algorithms needed by a general-purpose replication 
>>>>> scheme.
>>>>>
>>>>> Craig
>>>>>
>>>>> On Mar 9, 2006, at 9:48 AM, Jörg von Frantzius wrote:
>>>>>
>>>>>> Craig L Russell schrieb:
>>>>>>> Hi Jörg,
>>>>>>>
>>>>>>> There are no tests planned for this behavior.
>>>>>> That's good ;-)
>>>>>>>
>>>>>>> The issue is that it violates the contract of detachment. 
>>>>>>> Detachment is intended to provide a "long-running optimistic

>>>>>>> transaction" in which conflicts are detected in a subsequent

>>>>>>> transaction.
>>>>>> I'd find it a little sad if a great feature like easy replication

>>>>>> was sacrificed in favor of that. Unless replication should be 
>>>>>> reserved for JPOX (using a vendor extension), then maybe a future

>>>>>> version of the spec could have something along the lines of the 
>>>>>> solution described by Marco in 
>>>>>> http://www.jpox.org/servlet/jira/browse/CORE-2741
>>>>>>
>>>>>> That would be great.
>>>>>>
>>>>>> Just for completeness, and maybe it's just me, but the only 
>>>>>> sentence about detaching in general that I could find is
>>>>>> "These methods provide a way for an application to identify 
>>>>>> persistent instances, obtain
>>>>>> copies of these persistent instances, modify the detached 
>>>>>> instances either in the same JVM
>>>>>> or in a different JVM, apply the changes to the same or different

>>>>>> PersistenceManager,
>>>>>> and commit the changes."
>>>>>>
>>>>>> It's not really talking about an equivalent to long-running 
>>>>>> optimistic transactions, I find.
>>>>>>> If an instance is detached and then the underlying datastore

>>>>>>> instance is deleted, this is a consistency violation that should

>>>>>>> be detected by the transaction semantics. For example, in an

>>>>>>> order system, if a customer is in a long-running transaction

>>>>>>> with "groovy beads" in the shopping cart, and the administrators

>>>>>>> decide that "groovy beads" are no longer to be sold, you want

>>>>>>> the order that contains "groovy beads" to be rejected when the

>>>>>>> shopping cart arrives at checkout. You don't want that order
to 
>>>>>>> reinsert "groovy beads" into the database.
>>>>>> I agree that this surely must be catered for.
>>>>>>>
>>>>>>> Craig
>>>>>>>
>>>>>>> On Mar 9, 2006, at 8:40 AM, Jörg von Frantzius wrote:
>>>>>>>
>>>>>>>> Hi Craig,
>>>>>>>>
>>>>>>>> I was already afraid that "create a persistent instance"
might 
>>>>>>>> only apply to the PM cache, not the datastore (but only after

>>>>>>>> second read). However, would you say that JPOX is not JDO2

>>>>>>>> compliant if it created missing instances in the datastore

>>>>>>>> anyway? Will there be a test in the TCK2 that expects an

>>>>>>>> exception to be thrown if a detached instances does not exist

>>>>>>>> in the datastore?
>>>>>>>>
>>>>>>>> And, most of all, what sense would it make to forbid the

>>>>>>>> creation of missing detached instances in the datastore?
There 
>>>>>>>> is lots of application for that behaviour, and at least I
don't 
>>>>>>>> know of any problem with it.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Jörg
>>>>>>>>
>>>>>>>> Craig L Russell schrieb:
>>>>>>>>> Hi Jörg,
>>>>>>>>>
>>>>>>>>> On Mar 9, 2006, at 1:43 AM, Jörg von Frantzius wrote:
>>>>>>>>>
>>>>>>>>>> Craig L Russell schrieb:
>>>>>>>>>>>> Also I find it confusing that the method
most prominently 
>>>>>>>>>>>> used for inserting new objects shouldn't
do so for detached 
>>>>>>>>>>>> instances.
>>>>>>>>>>>
>>>>>>>>>>> There is a bunch of history that you should look
at, most of 
>>>>>>>>>>> which is in the jdo-dev archives. Bottom line,
we used to 
>>>>>>>>>>> have a different API, attachCopy, but we looked
at what it 
>>>>>>>>>>> had to do for transient and detached instances
and decided 
>>>>>>>>>>> that it wasn't worth making a different API for
attaching 
>>>>>>>>>>> detached instances.
>>>>>>>>>> That particular behaviour of attachCopy() wasn't
really 
>>>>>>>>>> specified, but it was pleasant JPOX-specific behaviour,
if I 
>>>>>>>>>> remember correctly. I saw the discussion and I didn't
see 
>>>>>>>>>> where inserting the instances would be forbidden
by the spec, 
>>>>>>>>>> and still I don't see where it says that, especially
in the 
>>>>>>>>>> light of 12.6.7. Please excuse my ignorance, where
does it 
>>>>>>>>>> say that?
>>>>>>>>>
>>>>>>>>> <spec>
>>>>>>>>> These methods make transient instances persistent and
apply 
>>>>>>>>> detached instance changes
>>>>>>>>> to the cache.
>>>>>>>>> ...
>>>>>>>>> For a detached instance, they locate or create a persistent
>>>>>>>>> instance with the same JDO identity as the detached instance,

>>>>>>>>> and merge the persistent
>>>>>>>>> state of the detached instance into the persistent instance.

>>>>>>>>> Only the state of persistent fields
>>>>>>>>> is merged.
>>>>>>>>> </spec>
>>>>>>>>>
>>>>>>>>> This means that if there is already a persistent instance
in 
>>>>>>>>> the cache with the same object id as the detached instance,

>>>>>>>>> the detached state will be merged. If there is not a

>>>>>>>>> persistent instance in the cache, a cache instance is
created 
>>>>>>>>> and the detached state is merged with the persistent
instance.
>>>>>>>>>
>>>>>>>>> But there is no creation aspect of makePersistent on
a 
>>>>>>>>> detached instance.
>>>>>>>>>
>>>>>>>>> Craig
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Craig
>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Craig
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mar 8, 2006, at 7:14 AM, Erik Bengtson
wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What happens when we invoke makePersistent
on a detached 
>>>>>>>>>>>>>> instance that was
>>>>>>>>>>>>>> deleted by another isolated process?
I suspect that we 
>>>>>>>>>>>>>> raise an exception
>>>>>>>>>>>>>> instead of reinserting it for a second
time. Is that right?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maybe this can be clarified in the
spec.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Craig Russell
>>>>>>>>>>>>> Architect, Sun Java Enterprise System

>>>>>>>>>>>>> http://java.sun.com/products/jdo
>>>>>>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Craig Russell
>>>>>>>>>>> Architect, Sun Java Enterprise System 
>>>>>>>>>>> http://java.sun.com/products/jdo
>>>>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Craig Russell
>>>>>>>>> Architect, Sun Java Enterprise System 
>>>>>>>>> http://java.sun.com/products/jdo
>>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Craig Russell
>>>>>>> Architect, Sun Java Enterprise System 
>>>>>>> http://java.sun.com/products/jdo
>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>
>>>>>>
>>>>>
>>>>> Craig Russell
>>>>> Architect, Sun Java Enterprise System 
>>>>> http://java.sun.com/products/jdo
>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>> P.S. A good JDO? O, Gasp!
>>>>>
>>>>
>>>
>>> Craig Russell
>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>> P.S. A good JDO? O, Gasp!
>>>
>>
>
> Craig Russell
>
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>
> 408 276-5638 mailto:Craig.Russell@sun.com
>
> P.S. A good JDO? O, Gasp!
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message