Mailing-List: contact jdo-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jdo-dev@db.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Date: Tue, 18 Apr 2006 11:50:33 -0700
From: Craig L Russell <Craig.Russell@Sun.COM>
Subject: Re: makePersistent detached instance deleted on database
In-reply-to: <44115F83.8030603@artnology.com>
Sender: Craig.Russell@Sun.COM
To: =?ISO-8859-1?Q?J=F6rg_von_Frantzius?= <joerg.von.frantzius@artnology.com>
Cc: jdo-dev@db.apache.org, jdo-experts-ext@Sun.COM
Message-id: <289C5CF4-BC2E-4FDA-8498-0610CDCE7800@Sun.COM>
MIME-version: 1.0
Content-type: multipart/signed; protocol="application/pkcs7-signature";
 boundary=Apple-Mail-28--424690312; micalg=sha1
References: <1141830841.440ef4b90c5dd@webmail.jpox.org>
 <B349573F-985A-47D0-8CA2-07E95F2FC591@Sun.COM>
 <440F8E98.7060505@artnology.com>
 <FC8CB0EB-8D56-4F18-823B-04C45E00508F@Sun.COM> <440FF8D5.30602@artnology.com>
 <8AEDB3E8-25D7-432F-A0C5-4667D3106C70@Sun.COM>
 <44105A80.6090501@artnology.com>
 <F98E0411-6275-4DFF-9B5A-7BFB9B1A283C@Sun.COM>
 <44106A64.8020607@artnology.com>
 <745FEFEF-F967-4AFA-804C-55E7EA2FC8FF@Sun.COM>
 <44115F83.8030603@artnology.com>

--Apple-Mail-28--424690312
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=ISO-8859-1;
	delsp=yes;
	format=flowed

Hi J=F6rg,

Sorry I lost this message in email limbo, but I am serious. Please =20
consider writing a proposal on adding methods to better support =20
replication. Two-paragraph email messages are not the level of detail =20=

we need to consider. How about a proposal?

Thanks,

Craig

On Mar 10, 2006, at 3:14 AM, J=F6rg von Frantzius wrote:

> Hi Craig,
>
> replication really is lost in a specification gap: makePersistent() =20=

> on transient instances won't update existing data, and on detached =20
> instances it won't insert new. For replication, you need both =20
> behaviours at the same time.
>
> That's really misfortunate for such a nice feature! Even more so as =20=

> it is not just theory, but it proves to be working in production =20
> with JPOX' old implementation of attachCopy().
>
> Regards,
> J=F6rg
>
> Craig L Russell schrieb:
>> Hi J=F6rg,
>>
>> Using detachment for replication is an interesting use case, and =20
>> I'd like to see more in-depth analysis of the issues that you =20
>> encounter once you've done with it.
>>
>> The use-case for detachment is long-running optimistic =20
>> transactions, as you have noted below. We did add makeTransient=20
>> (Object, useFetchPlan) as a way to disconnect objects from one =20
>> datastore that could be used with another, but I really doubt that =20=

>> we are going to be able to incorporate into the JDO API all the =20
>> policy algorithms needed by a general-purpose replication scheme.
>>
>> Craig
>>
>> On Mar 9, 2006, at 9:48 AM, J=F6rg von Frantzius wrote:
>>
>>> Craig L Russell schrieb:
>>>> Hi J=F6rg,
>>>>
>>>> There are no tests planned for this behavior.
>>> That's good ;-)
>>>>
>>>> The issue is that it violates the contract of detachment. =20
>>>> Detachment is intended to provide a "long-running optimistic =20
>>>> transaction" in which conflicts are detected in a subsequent =20
>>>> transaction.
>>> I'd find it a little sad if a great feature like easy replication =20=

>>> was sacrificed in favor of that. Unless replication should be =20
>>> reserved for JPOX (using a vendor extension), then maybe a future =20=

>>> version of the spec could have something along the lines of the =20
>>> solution described by Marco in http://www.jpox.org/servlet/jira/=20
>>> browse/CORE-2741
>>>
>>> That would be great.
>>>
>>> Just for completeness, and maybe it's just me, but the only =20
>>> sentence about detaching in general that I could find is
>>> "These methods provide a way for an application to identify =20
>>> persistent instances, obtain
>>> copies of these persistent instances, modify the detached =20
>>> instances either in the same JVM
>>> or in a different JVM, apply the changes to the same or different =20=

>>> PersistenceManager,
>>> and commit the changes."
>>>
>>> It's not really talking about an equivalent to long-running =20
>>> optimistic transactions, I find.
>>>> If an instance is detached and then the underlying datastore =20
>>>> instance is deleted, this is a consistency violation that should =20=

>>>> be detected by the transaction semantics. For example, in an =20
>>>> order system, if a customer is in a long-running transaction =20
>>>> with "groovy beads" in the shopping cart, and the administrators =20=

>>>> decide that "groovy beads" are no longer to be sold, you want =20
>>>> the order that contains "groovy beads" to be rejected when the =20
>>>> shopping cart arrives at checkout. You don't want that order to =20
>>>> reinsert "groovy beads" into the database.
>>> I agree that this surely must be catered for.
>>>>
>>>> Craig
>>>>
>>>> On Mar 9, 2006, at 8:40 AM, J=F6rg von Frantzius wrote:
>>>>
>>>>> Hi Craig,
>>>>>
>>>>> I was already afraid that "create a persistent instance" might =20
>>>>> only apply to the PM cache, not the datastore (but only after =20
>>>>> second read). However, would you say that JPOX is not JDO2 =20
>>>>> compliant if it created missing instances in the datastore =20
>>>>> anyway? Will there be a test in the TCK2 that expects an =20
>>>>> exception to be thrown if a detached instances does not exist =20
>>>>> in the datastore?
>>>>>
>>>>> And, most of all, what sense would it make to forbid the =20
>>>>> creation of missing detached instances in the datastore? There =20
>>>>> is lots of application for that behaviour, and at least I don't =20=

>>>>> know of any problem with it.
>>>>>
>>>>> Regards,
>>>>> J=F6rg
>>>>>
>>>>> Craig L Russell schrieb:
>>>>>> Hi J=F6rg,
>>>>>>
>>>>>> On Mar 9, 2006, at 1:43 AM, J=F6rg von Frantzius wrote:
>>>>>>
>>>>>>> Craig L Russell schrieb:
>>>>>>>>> Also I find it confusing that the method most prominently =20
>>>>>>>>> used for inserting new objects shouldn't do so for detached =20=

>>>>>>>>> instances.
>>>>>>>>
>>>>>>>> There is a bunch of history that you should look at, most of =20=

>>>>>>>> which is in the jdo-dev archives. Bottom line, we used to =20
>>>>>>>> have a different API, attachCopy, but we looked at what it =20
>>>>>>>> had to do for transient and detached instances and decided =20
>>>>>>>> that it wasn't worth making a different API for attaching =20
>>>>>>>> detached instances.
>>>>>>> That particular behaviour of attachCopy() wasn't really =20
>>>>>>> specified, but it was pleasant JPOX-specific behaviour, if I =20
>>>>>>> remember correctly. I saw the discussion and I didn't see =20
>>>>>>> where inserting the instances would be forbidden by the spec, =20=

>>>>>>> and still I don't see where it says that, especially in the =20
>>>>>>> light of 12.6.7. Please excuse my ignorance, where does it =20
>>>>>>> say that?
>>>>>>
>>>>>> <spec>
>>>>>> These methods make transient instances persistent and apply =20
>>>>>> detached instance changes
>>>>>> to the cache.
>>>>>> ...
>>>>>> For a detached instance, they locate or create a persistent
>>>>>> instance with the same JDO identity as the detached instance, =20
>>>>>> and merge the persistent
>>>>>> state of the detached instance into the persistent instance. =20
>>>>>> Only the state of persistent fields
>>>>>> is merged.
>>>>>> </spec>
>>>>>>
>>>>>> This means that if there is already a persistent instance in =20
>>>>>> the cache with the same object id as the detached instance, =20
>>>>>> the detached state will be merged. If there is not a =20
>>>>>> persistent instance in the cache, a cache instance is created =20
>>>>>> and the detached state is merged with the persistent instance.
>>>>>>
>>>>>> But there is no creation aspect of makePersistent on a =20
>>>>>> detached instance.
>>>>>>
>>>>>> Craig
>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Craig
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Craig
>>>>>>>>>>
>>>>>>>>>> On Mar 8, 2006, at 7:14 AM, Erik Bengtson wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> What happens when we invoke makePersistent on a detached =20
>>>>>>>>>>> instance that was
>>>>>>>>>>> deleted by another isolated process? I suspect that we =20
>>>>>>>>>>> raise an exception
>>>>>>>>>>> instead of reinserting it for a second time. Is that right?
>>>>>>>>>>>
>>>>>>>>>>> Maybe this can be clarified in the spec.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Craig Russell
>>>>>>>>>> Architect, Sun Java Enterprise System http://java.sun.com/=20
>>>>>>>>>> products/jdo
>>>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Craig Russell
>>>>>>>> Architect, Sun Java Enterprise System http://java.sun.com/=20
>>>>>>>> products/jdo
>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Craig Russell
>>>>>> Architect, Sun Java Enterprise System http://java.sun.com/=20
>>>>>> products/jdo
>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>
>>>>>
>>>>
>>>> Craig Russell
>>>> Architect, Sun Java Enterprise System http://java.sun.com/=20
>>>> products/jdo
>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>> P.S. A good JDO? O, Gasp!
>>>>
>>>
>>
>> Craig Russell
>> Architect, Sun Java Enterprise System http://java.sun.com/products/=20=

>> jdo
>> 408 276-5638 mailto:Craig.Russell@sun.com
>> P.S. A good JDO? O, Gasp!
>>
>


--Apple-Mail-28--424690312
Content-Transfer-Encoding: base64
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Disposition: attachment;
	filename=smime.p7s

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGUDCCAwkw
ggJyoAMCAQICEB7Y7fLHmKoeHTUdbtYrvpAwDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UEBhMCWkEx
JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ
ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA1MTIxNTIxNDc0NFoXDTA2MTIxNTIxNDc0
NFowbDEQMA4GA1UEBBMHUnVzc2VsbDEUMBIGA1UEKhMLQ3JhaWcgTGFpcmQxHDAaBgNVBAMTE0Ny
YWlnIExhaXJkIFJ1c3NlbGwxJDAiBgkqhkiG9w0BCQEWFUNyYWlnLlJ1c3NlbGxAU3VuLkNPTTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMXXgEAm88nu8cFbxXSWqTq+GaYpCx/0QSom
8kBbVxpJIURuO4ErLDupeXu6y9+5e7ZKLbeSQE5xbfYPcQR/IhqmYKy5YqRcuzPXREBj2zKEkZph
pNXWpHgMdx9W8dq0Cu2i9Ux/S6c2MuEHrP6gfGGll+b/mzLoO280QHTuE4pcpKntRnwZdGxQ/5l8
IL+eLP+jpJAbYW9C+KNKofZtS6V6R0uzlqTOsEdZvwxZQ4mmPgHoz1+Gjwme/PC5sKvF09MaJDiI
pj9SvZ4CTCgcDZV78J086YwlVbMC0VQotjhu1p42lr8CS33IXLz3OWNrDETCAepah/Dgw2ZZApQ9
9L0CAwEAAaMyMDAwIAYDVR0RBBkwF4EVQ3JhaWcuUnVzc2VsbEBTdW4uQ09NMAwGA1UdEwEB/wQC
MAAwDQYJKoZIhvcNAQEEBQADgYEAKdIkgAWCg2Bi7ocnstfJA4iymTRI2/L4oQx9zvllM9bNJ2cR
cecJIx3HuoHbhPvemh1GExEPgHU+dXSxDmD0BEmPnhSReKCURyslnbMphPZ5kR6USzQFrRa+v0ii
J+SBO9VQYTQWT+xEjmRLM76MfkBFw3IOC9CUkRoYZ88pOoUwggM/MIICqKADAgECAgENMA0GCSqG
SIb3DQEBBQUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQH
EwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZp
Y2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1h
aWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMw
NzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3Rl
IENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWls
IElzc3VpbmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUE
cJ3f6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH5/Ef
kTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBlyYLf7AgMB
AAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJodHRwOi8vY3Js
LnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNVHQ8EBAMCAQYwKQYD
VR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqGSIb3DQEBBQUAA4GB
AEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQcUCCTcDz9reFhYsPZOhl+
hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bGCE6u9uo05RAaWzVNd+NWIXiC
3CEZNd4ksdMdRv9dX2VPMYIDEDCCAwwCAQEwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh
d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt
YWlsIElzc3VpbmcgQ0ECEB7Y7fLHmKoeHTUdbtYrvpAwCQYFKw4DAhoFAKCCAW8wGAYJKoZIhvcN
AQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDYwNDE4MTg1MDM0WjAjBgkqhkiG9w0B
CQQxFgQUkm8yrec5vExi+DtOe0ktHpCAF/8wgYUGCSsGAQQBgjcQBDF4MHYwYjELMAkGA1UEBhMC
WkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0
ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhAe2O3yx5iqHh01HW7WK76QMIGHBgsqhkiG
9w0BCRACCzF4oHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQ
dHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhAe
2O3yx5iqHh01HW7WK76QMA0GCSqGSIb3DQEBAQUABIIBAJYPMXLrZ6jKNHB6VkVIWKMW7A30/8+j
4K3sM/PpwpGx6fH9W16Tvd+jtzyVOfzQbf2YWxZuvT+4eln7y9dD45oybQUS4OF7WctG1mFs0cee
rYYopzJ6iQ+5Vbs19g4uRmAGnU7sY0CLBvuB/m2oOAy/H3ZvzVBvsumrUOPdr8oOcNzvLNjqdLtz
6bBJ6en2O8rnYiKs+xCEQP5zlTgmK3VSg+pRaE0X/f+m5eULtwQmujqN7KLpDDSPxMGErYdnCHFp
b83o8TgpBFAV2si2CSfv9VYzG9MLxlQN4pS8c8fQNETf36Yy8p5RWZ//mNjAUXfe/PU9pK2YmDXe
6/4Sip8AAAAAAAA=

--Apple-Mail-28--424690312--