Mailing-List: contact jdo-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jdo-dev@db.apache.org
Received-SPF: pass (herse.apache.org: local policy)
Date: Fri, 13 Apr 2007 11:36:10 -0700
From: Craig L Russell <Craig.Russell@Sun.COM>
Subject: Re: enlistment of native connections for optimistic txs
In-reply-to: <461B52EE.6080609@objectmagic.org>
Sender: Craig.Russell@Sun.COM
To: Apache JDO project <jdo-dev@db.apache.org>,
        JDO Expert Group <jdo-experts-ext@Sun.COM>
Message-id: <71CDF8BA-E291-4AB9-9E36-F24F44BA13FE@SUN.com>
MIME-version: 1.0
Content-type: multipart/signed; protocol="application/pkcs7-signature";
 boundary=Apple-Mail-10-613674902; micalg=sha1
References: <1176065353.461955493a030@webmail.jpox.org>
 <5E68EE17-2CA0-4D25-8326-358DE42BF846@SUN.com>
 <461B52EE.6080609@objectmagic.org>

--Apple-Mail-10-613674902
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=ISO-8859-1;
	delsp=yes;
	format=flowed

Hi Guido,

A new proposal is attached below.

On Apr 10, 2007, at 2:03 AM, Guido Anzuoni wrote:

> Craig L Russell wrote:
>> Hi Erik,
>>
>> Thanks for clarifying your proposal.
>>
>> I agree that this improves portability, but with a penalty. You =20
>> can no
>> longer use optimistic concurrency once you decide to do some native
>> SQL, even if you don't ever perform any INSERT, DELETE, or UPDATE
>> statements. And this will have negative performance consequences for
>> some applications.
>>
>> While backward compatibility is important, I would agree with you =20
>> that
>> correctness is even more important. So it doesn't bother me too much
>> if we change the semantic so that by default, getting a datastore
>> connection is equivalent to doing a flush in an optimistic
>> transaction. But I'd still like to give the user the ability to get
>> the same level of performance they now enjoy by deferring the
>> enlistment in the datastore transaction to flush or commit.
>>
>> So how about adding a flag that defaults to true if not specified to
>> enlist the datastore connection in the datastore transaction?
>>
>> <proposed>
>> Connection getDataStoreConnection()
>>
>> This method is equivalent to getDataStoreConnection(true).

Seems like the default should be different based on whether you are =20
in a JDO transaction.

So maybe the default should be getDataStoreConnection=20
(pm.currentTransaction().isActive())
>>
>> Connection getDataStoreConnection(boolean enlist)
>>
>> If this method is called outside an active transaction, the object
>> returned will not be enlisted in any
>> transaction, regardless of the setting of the enlist parameter.
> In this case, the user does not know that his request has not been =20
> honored.

Maybe it makes sense to throw a JDOUserException, since there is no =20
transaction in which to enlist the connection, and the default is =20
false. A user can only get the exception by asking for something that =20=

is impossible.
>
>>
>> If it is called while a datastore transaction is active, the object
>> returned will be enlisted in the current transaction,  regardless of
>> the setting of the enlist parameter.
>>
> Again, the implementation might do something different from what =20
> requested.

Right, either a non-enlisted connection (enlist=3D=3Dfalse) or the =20
connection enlisted in the current transaction.

The use case for this is to grab a non-enlisted connection that is =20
used e.g. to get a sequence number that will never be rolled back.
>
>> If it is called while an optimistic transaction is active, and the
>> enlist parameter is true, a datastore transaction is begun if not
>> already begun, and the object returned will be enlisted in the =20
>> current
>> datastore transaction.
>>
>> If it is called while an optimistic transaction is active, the enlist
>> parameter is false, and a datastore transaction is already begun, the
>> object returned will be enlisted in the datastore transaction.
> Same as above

Right, either a separate non-enlisted connection (enlist=3D=3Dfalse) or =20=

the connection enlisted in the current transaction (enlist=3D=3Dtrue).

>
>> If it is called while an optimistic transaction is active, the enlist
>> parameter is false, and a datastore transaction is not already begun,
>> the object returned will not be enlisted in any datastore =20
>> transaction.
>> </proposed>
> What if the user issues some INSERT, DELETE, UPDATE ?

This is the user's choice, which is not a default behavior, but =20
something explicitly asked for. The default will always be to get an =20
enlisted connection, and the only way to get a non-enlisted =20
connection is to ask for it.

> Is it possible to call conn.commit() ?

Yes, the non-enlisted connection changes were already committed (auto-=20=

commit). We should add this to the spec, that the non-enlisted =20
connection is auto-commit.
>
> I agree that there is a potential penalty in optimistic =20
> transactions if
> we blindly enlist native connections but, if we let the user know
> exactly what is going on,
> an aware decision can be taken.
> getDataStoreConnection() will always return the enlisted connection if
> any, otherwise null is returned (maybe getNativeConnection() could
> return null).

I think that returning null is never the right behavior.

> Now, if the user wants to access a consistent backend, he HAVE to
> flush(), triggering connection enlistment (what if there are no =20
> pending
> updates ?).
> If the user wants to operate on an independent connection to the same
> datastore used by the PM he could use what is configured in the PMF
> (OK, some helper methods might...help).

I think that inventing a parallel mechanism is overkill if we can =20
give users what the need from this one.

> It could be also achieved with a different handling of the proposed
> enlist flag for the 'false' scenarios, but it would be strange to
> request to the
> PM a datastore connection the PM is not using at all !!!. It would =20
> look
> like a data source.

Yes, that's the intent. There is no need for the user to somehow go =20
behind the scenes to figure out how the PMF is configured and get a =20
connection from that same data source.

Craig

<proposed>
Connection getDataStoreConnection()

This method is equivalent to getDataStoreConnection=20
(pm.currentTransaction().isActive()). That is, it defaults to =20
getDataStoreConnection(true) is a JDO transaction is active, and =20
getDataStoreConnection(false) if not.

JDOConnection getDataStoreConnection(boolean enlist)

This method returns either a JDOConnection whose underlying native =20
connection is enlisted in the native datastore transaction in use by =20
the persistence manager; or a JDOConnection whose underlying native =20
connection is not enlisted in any native transaction. An unenlisted =20
connection is set to auto-commit if using a data source that has this =20=

concept (e.g. JDBC).

If this method is called outside an active transaction, with the =20
enlist parameter false, the object returned will not be enlisted in =20
any transaction. If this method is called outside an active =20
transaction, with the enlist parameter true, JDOUserException is thrown.

If this method is called while a datastore transaction is active, =20
with the enlist parameter false, the object returned will not be =20
enlisted in any transaction. If this method is called while a =20
datastore transaction is active, with the enlist parameter true, the =20
object returned will be enlisted in the current native datastore =20
transaction.

If this method is called while an optimistic transaction is active, =20
with the enlist parameter false, the object returned will not be =20
enlisted in any transaction. If this method is called while an =20
optimistic transaction is active, and the enlist parameter is true, a =20=

native datastore transaction is begun if not already begun, and the =20
object returned will be enlisted in the current native datastore =20
transaction.

</proposed>

Craig

>
> Guido.
>>
>> Craig
>>
>> On Apr 8, 2007, at 1:49 PM, Erik Bengtson wrote:
>>
>>>
>>> Hi,
>>>
>>> This is a change proposal to the spec with regards to enlistment of
>>> native
>>> connections when optimistic transactions are used.
>>>
>>> The enlistment of native connections into a JDO transaction is
>>> conditioned to
>>> the fact that a flush call has been performed before the =20
>>> connection is
>>> obtained.
>>>
>>> The flush call is a JDO implementation decision which can differ =20
>>> between
>>> different implementations. See the example:
>>>
>>> 1. tx.begin();
>>> 2. //more operations here....
>>> 3. JDOConnection conn =3D pm.getDataStoreConnection();
>>> 4. Connection sqlconn =3D (Connection) conn.getNativeConnection();
>>> 5. sqlconn.execute("DELETE FROM ANIMAL WHERE NAME =3D 'CAT'")
>>> 5. conn.close();
>>> 6. tx.rollback();
>>>
>>> There are two possible behaviors:
>>>
>>> - The JDO implementation decides to perform a flush between 1 and =20=

>>> 3: the
>>> connection is enlisted, and DELETE all CATS from ANIMAL is rolled =20=

>>> back
>>> - The JDO implementation decides to NOT perform a flush between 1 =20=

>>> and
>>> 3: the
>>> connection is not enlisted, and DELETE all CATS from ANIMAL is not
>>> rolled back
>>>
>>> As you can see, this is not portable and not ACID, so I would =20
>>> like to
>>> propose
>>> that enlistment of connections is done whenever a transaction is =20
>>> opened.
>>>
>>> The change is to the following paragraph:
>>>
>>> The JDO spec =A712.16 - getDataStoreConnection:
>>>
>>> "If this method is called while a datastore transaction is active,
>>> the object
>>> returned will be enlisted in the current transaction. If called =20
>>> in an
>>> optimistic transaction before flush has been called, or outside an
>>> active
>>> transaction, the object returned will not be enlisted in any
>>> transaction."
>>>
>>> AS:
>>>
>>> "If this method is called while a datastore or optimistc transaction
>>> is active,
>>> the object returned will be enlisted in the current transaction. If
>>> called
>>> outside an active transaction, the object returned will not be
>>> enlisted in any
>>> transaction."
>>>
>>>
>>
>> Craig Russell
>> Architect, Sun Java Enterprise System http://java.sun.com/products/=20=

>> jdo
>> 408 276-5638 mailto:Craig.Russell@sun.com
>> P.S. A good JDO? O, Gasp!
>>
>

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!


--Apple-Mail-10-613674902
Content-Transfer-Encoding: base64
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Disposition: attachment;
	filename=smime.p7s

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGUDCCAwkw
ggJyoAMCAQICECpJVMO68ii+Xfsc1O1YYFIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx
JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ
ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MTIwOTE5NTEwNVoXDTA3MTIwOTE5NTEw
NVowbDEQMA4GA1UEBBMHUnVzc2VsbDEUMBIGA1UEKhMLQ3JhaWcgTGFpcmQxHDAaBgNVBAMTE0Ny
YWlnIExhaXJkIFJ1c3NlbGwxJDAiBgkqhkiG9w0BCQEWFUNyYWlnLlJ1c3NlbGxAU3VuLkNPTTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMNAB4Ih+ShsCt89HzqIdwEx8L4o1UHiY6V7
16mrCedfd4Y0/uI7z9Zam8ysgEh+F7aDnQEKmEsVFN35G4nPMfLU6dZYkvADwUjbq82t/dJ3FDDg
Q945nHHpqECZff/S/UMho9AFfj6PZvZBAlDCJAayb4RdKIlfuvPW9YcQStQ1IfVJcVuKnC0Q+tdc
a4A7zn7IzLOQohO1lTc3hXSBigEIGiGYn6Ny0wmexfA3X1WsXekFx5czd+M4GjDjswn8CNoBmnBr
jOTGK1mOsXR6GSRHnly2s9xTdE4qv9qimM+7C2yzMHbKcszV7OQoLsRsZKDh+6u9wYU+TrjcY4ym
bA8CAwEAAaMyMDAwIAYDVR0RBBkwF4EVQ3JhaWcuUnVzc2VsbEBTdW4uQ09NMAwGA1UdEwEB/wQC
MAAwDQYJKoZIhvcNAQEFBQADgYEAU/EpPDztnb55Fz7iGSVm1mYEVj5m2OQKTYG26POUAomCBRrt
/CdBBvqYmcHUTpra0qLELHAQadYFl2v11iQkqwF5PPJs19oU/zA0m5qFnOMTAiCvel7IprIwA2r6
eJR9siaPwDRgVJ/Sj71dD+utwf+nRrNy0/7PMNK5y+ocsYQwggM/MIICqKADAgECAgENMA0GCSqG
SIb3DQEBBQUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQH
EwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZp
Y2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1h
aWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMw
NzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3Rl
IENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWls
IElzc3VpbmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUE
cJ3f6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH5/Ef
kTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBlyYLf7AgMB
AAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJodHRwOi8vY3Js
LnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNVHQ8EBAMCAQYwKQYD
VR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqGSIb3DQEBBQUAA4GB
AEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQcUCCTcDz9reFhYsPZOhl+
hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bGCE6u9uo05RAaWzVNd+NWIXiC
3CEZNd4ksdMdRv9dX2VPMYIDEDCCAwwCAQEwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh
d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt
YWlsIElzc3VpbmcgQ0ECECpJVMO68ii+Xfsc1O1YYFIwCQYFKw4DAhoFAKCCAW8wGAYJKoZIhvcN
AQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDcwNDEzMTgzNjEwWjAjBgkqhkiG9w0B
CQQxFgQUvJMNEQeNsB1ZD7X7vYpAASNphSswgYUGCSsGAQQBgjcQBDF4MHYwYjELMAkGA1UEBhMC
WkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0
ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhAqSVTDuvIovl37HNTtWGBSMIGHBgsqhkiG
9w0BCRACCzF4oHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQ
dHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhAq
SVTDuvIovl37HNTtWGBSMA0GCSqGSIb3DQEBAQUABIIBAMAWCvCFVrZG78wr3basDOpSP/LWy3VM
qr0pBg7bF4tfCdx0EwvBsZIPZtzCVA4nG5+ZLQZ5AhUgYRlDub4Gn0I5N/pznwVfBnQ2ytn5Booe
VVmj/GD3kn90Fd8GbqPxtrgWsfpAtlYUw4BvldnYJR5gSgg3+w7gkECm1zYcuMmB0bPKxronMYkA
ZMsSYgCBb/43CH/LrnUTGQhfKqHysSl3S52ylxSNUzoF/u0IHOTulA3h/LSv0w/7QYxHs/5R3s7v
ho0zu2YBvl1Ef0IZbzRAucTWFc36aC4kRts442wahJ+jve6p/YLQmng+7EhBAXIfaK96V+UQUHOR
JfUHXukAAAAAAAA=

--Apple-Mail-10-613674902--