Return-Path: Delivered-To: apmail-db-jdo-dev-archive@www.apache.org Received: (qmail 27228 invoked from network); 19 Sep 2005 17:51:58 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 19 Sep 2005 17:51:58 -0000 Received: (qmail 16590 invoked by uid 500); 19 Sep 2005 17:51:56 -0000 Mailing-List: contact jdo-dev-help@db.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jdo-dev@db.apache.org Delivered-To: mailing list jdo-dev@db.apache.org Received: (qmail 16561 invoked by uid 99); 19 Sep 2005 17:51:55 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Sep 2005 10:51:55 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=HTML_MESSAGE X-Spam-Check-By: apache.org Received: from [192.18.98.34] (HELO brmea-mail-3.sun.com) (192.18.98.34) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Sep 2005 10:52:03 -0700 Received: from fe3.sun.com (fe3.Sun.COM [192.18.108.80] (may be forged)) by brmea-mail-3.sun.com (8.12.10/8.12.9) with ESMTP id j8JHpp1L023878 for ; Mon, 19 Sep 2005 11:51:51 -0600 (MDT) Received: from conversion-daemon.fe3.sun.com by fe3.sun.com (Sun Java System Messaging Server 6.1 HotFix 0.02 (built Aug 25 2004)) id <0IN200B01R91W400@fe3.sun.com> (original mail from Craig.Russell@Sun.COM) for jdo-dev@db.apache.org; Mon, 19 Sep 2005 11:51:51 -0600 (MDT) Received: from [129.145.133.145] by fe3.sun.com (Sun Java System Messaging Server 6.1 HotFix 0.02 (built Aug 25 2004)) with ESMTPSA id <0IN200MHCSAEBTJ6@fe3.sun.com> for jdo-dev@db.apache.org; Mon, 19 Sep 2005 11:51:50 -0600 (MDT) Date: Mon, 19 Sep 2005 10:51:49 -0700 From: Craig Russell Subject: Re: connection pool settings for c3p0 In-reply-to: <432EF155.5040209@spree.de> To: jdo-dev@db.apache.org Message-id: <08184EC2-DC12-4150-A046-5CC8E6B95BE9@Sun.COM> MIME-version: 1.0 X-Mailer: Apple Mail (2.734) Content-type: multipart/signed; protocol="application/pkcs7-signature"; boundary=Apple-Mail-63-668738614; micalg=sha1 References: <06E3EF9374199F4F9EB39DE722CD54D43E58B3@posylnyi.learnquest.net> <432EB9B7.3050009@spree.de> <18633D06-CC1F-4780-A6B1-C94FDC1D2DC5@Sun.COM> <432EEBC2.5080209@sun.com> <4AB4BD5F-F907-4D2A-80A0-2DCF2ED9C48D@Sun.COM> <432EF155.5040209@spree.de> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N --Apple-Mail-63-668738614 Content-Type: multipart/alternative; boundary=Apple-Mail-62-668737960 --Apple-Mail-62-668737960 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi Michael, On Sep 19, 2005, at 10:11 AM, Michael Watzek wrote: > Hi Craig, > > >> Hi, >> The reason to close the pmf is to gracefully close the underlying >> files. This was a requirement of the original FOStore >> implementation, and I don't remember exactly why we wanted to do >> that, but we did. >> > I recall the same. If the files are not closed, then the file > descriptors are not released (unless the OS releases them > implicitly at application shutdown). The FOStore implementation registers a shutdown hook with the VM so it can clean up. I recall this was added after the TCK tests were all done. > > >> It's worth experimenting with this to see if we can avoid closing >> the PMF in case we are running multiple tests. We might need a >> maven postGoal to close the pmf after all the tests have run. >> > That would start another VM, right? Doh! > I think, we need to close before VM shutdown. We can implement a > shutdown hook, or we close PMF in BatchTestRunner. This sounds like the right answer. BTR knows when all the tests in the VM have been run. Karan, is this enough information for you to take a look at improvement? Thanks, Craig > > Regards, > Michael > >> Craig >> On Sep 19, 2005, at 9:48 AM, Michelle Caisse wrote: >> >>> There are some tests in api.persistencemanagerfactory that close >>> or attempt to close the pmf, so that would have to be tested for. >>> >>> A new Properties is passed with each invocation of >>> BatchTestRunner by maven, that is with each separate >>> configuration. However, some tests set pmf properties. In those >>> cases, the pmf would have to be closed and a new one created for >>> the next test. >>> >>> -- Michelle >>> >>> Craig Russell wrote: >>> >>> >>> >>>> Hi, >>>> >>>> It might be worthwhile looking at the JDO_Test method where the >>>> PMF is acquired. Perhaps a static Map that holds the PMF between >>>> tests would work. The thing to watch for is to make sure that >>>> the PMF that's cached fully meets the requirements of the >>>> Properties/Map that is being used. There is some logic there. >>>> >>>> I think that for most cases in the tck, the identical Properties >>>> is being passed in and maybe all we need is to have JDO_Test >>>> make a copy of the PMF properties and a reference to the PMF and >>>> if the properties matches the cached Properties, return the >>>> cached PMF. >>>> >>>> We need to make sure that this doesn't interfere with the PMF >>>> that uses a different Properties by design. >>>> >>>> Craig >>>> >>>> On Sep 19, 2005, at 6:14 AM, Michael Watzek wrote: >>>> >>>> >>>> >>>>> Hi Karan, >>>>> >>>>> >>>>> >>>>> >>>>>> Hi Michael, >>>>>> I have noticed one thing in the tests. The BatchTestRunner >>>>>> creates a new object for each TestCase and then calls the test >>>>>> () method. Since each one of them inherits the pmf instance >>>>>> variable, dont you think that with each test we are trying to >>>>>> get a new pmf and that initiates creation of a new pool of >>>>>> connections. This might be the reason connection pooling is >>>>>> slower than running tests without connection pooling. That 10% >>>>>> of overhead when we run tests with connection pooling, is in >>>>>> my opinion the time taken to create that pool (thats what it >>>>>> looks like when i looked at the JPOX RDBMS log earlier). >>>>>> >>>>>> >>>>>> >>>>> Right! Each test method calls the JDO implementation returning >>>>> a pmf instance. Finally, the test method calls pmf.close() and >>>>> nullifies the variable holding the pmf instance. >>>>> >>>>> Provided, that each getPMF() call creates a new pmf instance, >>>>> then we do not benefit from connetion pooling because the pool >>>>> is bound to a pmf instance (as you mentioned above). That's >>>>> what we see when we run TCK20 with JPOX. >>>>> >>>>> However, a JDO implementation may return the same pmf instance >>>>> if you pass the same properties to getPMF in subsequent calls. >>>>> Such implementations would benefit from connection pooling when >>>>> they are called by TCK20. >>>>> >>>>> Regards, >>>>> Michael >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> I might be wrong, because i havent had the time to understand >>>>>> the JDO_Test class fully. If I am wrong, then I think it might >>>>>> be worth a try to increase the connection pool size a little >>>>>> bit, just to see if it speeds up the tests. >>>>>> I will try it and let you know. Thanks for the info on c3p0 >>>>>> properties. >>>>>> Karan Singh >>>>>> Senior Technical Consultant >>>>>> Learnquest >>>>>> -----Original Message----- >>>>>> From: Michael Watzek [mailto:mwa.tech@spree.de] >>>>>> Sent: Mon 9/19/2005 8:26 AM >>>>>> To: jdo-dev@db.apache.org >>>>>> Subject: Re: connection pool settings for c3p0 >>>>>> Hi Karan, >>>>>> up to now there is no file "c3p0.properties". Thus, we run >>>>>> C3P0 using the default configuration. Do you think we can >>>>>> benefit from changing the default? The right place for such a >>>>>> file would be under test/conf. >>>>>> Regards, >>>>>> Michael >>>>>> >>>>>> >>>>>> >>>>>>> Where are we doing connection pool settings for c3p0? I >>>>>>> couldnt find the c3p0.properties file anywhere >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>>> -- >>>>> ------------------------------------------------------------------ >>>>> - >>>>> Michael Watzek Tech@Spree Engineering GmbH >>>>> mailto:mwa.tech@spree.de Buelowstr. 66 >>>>> Tel.: ++49/30/235 520 36 10783 Berlin - Germany >>>>> Fax.: ++49/30/217 520 12 http://www.spree.de/ >>>>> ------------------------------------------------------------------ >>>>> - >>>>> >>>>> >>>>> >>>> >>>> Craig Russell >>>> >>>> Architect, Sun Java Enterprise System http://java.sun.com/ >>>> products/jdo >>>> >>>> 408 276-5638 mailto:Craig.Russell@sun.com >>>> >>>> P.S. A good JDO? O, Gasp! >>>> >>>> >>>> >>>> >>> >>> >>> >> Craig Russell >> Architect, Sun Java Enterprise System http://java.sun.com/products/ >> jdo >> 408 276-5638 mailto:Craig.Russell@sun.com >> P.S. A good JDO? O, Gasp! >> > > > -- > ------------------------------------------------------------------- > Michael Watzek Tech@Spree Engineering GmbH > mailto:mwa.tech@spree.de Buelowstr. 66 > Tel.: ++49/30/235 520 36 10783 Berlin - Germany > Fax.: ++49/30/217 520 12 http://www.spree.de/ > ------------------------------------------------------------------- > Craig Russell Architect, Sun Java Enterprise System http://java.sun.com/products/jdo 408 276-5638 mailto:Craig.Russell@sun.com P.S. A good JDO? O, Gasp! --Apple-Mail-62-668737960 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hi Michael,

On = Sep 19, 2005, at 10:11 AM, Michael Watzek wrote:

Hi Craig,


The reason to close the pmf is = to gracefully close the underlying files. This was a requirement of the = original FOStore implementation, and I don't remember exactly why we = wanted to do that, but we did.

I recall the same. If the files are not closed, then = the file descriptors are not released (unless the OS releases them = implicitly at application shutdown).

The FOStore implementation = registers a shutdown hook with the VM so it can clean up. I recall this = was added after the TCK tests were all done.

=
It's worth = experimenting with this to see if we can avoid closing the PMF in case = we are running multiple tests. We might need a maven postGoal to close = the pmf after all the tests have run.

That would start another VM, right? =

Doh!=A0


I think, we need to close before VM shutdown. We can = implement a shutdown hook, or we close PMF in = BatchTestRunner.

This sounds like the right = answer. BTR knows when all the tests in the VM have been = run.

Karan, = is this enough information for you to take a look at = improvement?

Thanks,

Craig


Regards,
Michael
=
On Sep 19, 2005, at 9:48 AM, = Michelle Caisse wrote:

There are some tests in = api.persistencemanagerfactory that close or attempt to close the pmf, so = that would have to be tested for.

A new Properties is passed with = each invocation of BatchTestRunner by maven, that is with each separate = configuration.=A0 However, = some tests set pmf properties.=A0 = In those cases, the pmf would have to be closed and a new one = created for the next test.

-- Michelle

Craig = Russell wrote:



Hi,

It might be = worthwhile looking at the JDO_Test method where the PMF is acquired. = Perhaps a static Map that holds the PMF between tests would work. The = thing to watch for is to make sure that the PMF that's cached fully = meets the requirements of the Properties/Map that is being used. There = is some logic there.
I think that for most cases in = the tck, the identical Properties is being passed in and maybe all we = need is to have JDO_Test make a copy of the PMF properties and a = reference to the PMF and if the properties matches the cached = Properties, return the cached PMF.

We need to make sure that this = doesn't interfere with the PMF that uses a different Properties by = design.

Craig

On Sep 19, 2005, at 6:14 AM, = Michael Watzek wrote:

=
Hi = Karan,




Hi Michael,
I have = noticed one thing in the tests. The BatchTestRunner creates a new object = for each TestCase and then calls the test() method. Since each one of = them inherits the pmf instance variable, dont you think that with each = test we are trying to get a new pmf and that initiates creation of a new = pool of connections. This might be the reason connection pooling is = slower than running tests without connection pooling. That 10% of = overhead when we run tests with connection pooling, is in my opinion the = time taken to create that pool (thats what it looks like when i looked = at the JPOX RDBMS log earlier).


=
Right! Each test method calls = the JDO implementation returning a pmf instance. Finally, the test = method calls pmf.close() and nullifies the variable holding the pmf = instance.

Provided, that each getPMF() call creates a new pmf = instance, then we do not benefit from connetion pooling because the pool = is bound to a pmf instance (as you mentioned above). That's what we see = when we run TCK20 with JPOX.

However, a JDO implementation = may return the same pmf instance if you pass the same properties to = getPMF in subsequent calls. Such implementations would benefit from = connection pooling when they are called by TCK20.

Michael




=
I might be = wrong, because i havent had the time to understand the JDO_Test class = fully. If I am wrong, then I think it might be worth a try to increase = the connection pool size a little bit, just to see if it speeds up the = tests.
I will try it and let you know. = Thanks for the info on c3p0 properties.
=A0Karan Singh
Senior Technical Consultant
Learnquest
-----Original = Message-----
From: Michael Watzek [mailto:mwa.tech@spree.de]
Sent: Mon 9/19/2005 8:26 AM
To: jdo-dev@db.apache.org <mailto:jdo-dev@db.apache.org>=
Subject: Re: connection pool settings for = c3p0
=A0Hi Karan,
up to now there is no file "c3p0.properties". Thus, = we run C3P0 using the default configuration. Do you think we can benefit = from changing the default? The right place for such a file would be = under test/conf.
Regards,
Michael


=
Where are we = doing connection pool settings for c3p0? I couldnt find the = c3p0.properties file anywhere



=


--=A0
Michael Watzek=A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Tech@Spree Engineering = GmbH
mailto:mwa.tech@spree.de=A0 =A0 =A0 =A0 Buelowstr. = 66
Tel.:=A0 = ++49/30/235 520 36 =A0 =A0 = =A0 10783 Berlin - Germany
Fax.:=A0 ++49/30/217 520 12 =A0 =A0 =A0 http://www.spree.de/




Craig = Russell

Architect, Sun Java Enterprise System http://java.sun.com/products/jdo=

408 276-5638 mailto:Craig.Russell@sun.com

P.S. A = good JDO? O, Gasp!






Craig = Russell
Architect, Sun Java Enterprise = System http://java.sun.com/products/jdo=
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
=


--=A0
Michael Watzek=A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Tech@Spree Engineering = GmbH
mailto:mwa.tech@spree.de=A0 =A0 =A0 =A0 Buelowstr. = 66
Tel.:=A0 = ++49/30/235 520 36 =A0 =A0 = =A0 10783 Berlin - Germany
Fax.:=A0 ++49/30/217 520 12 =A0 =A0 =A0 http://www.spree.de/

=

Craig Russell

Architect, Sun Java = Enterprise System http://java.sun.com/products/jdo=

408 = 276-5638 mailto:Craig.Russell@sun.com

P.S. A = good JDO? O, Gasp!


= --Apple-Mail-62-668737960-- --Apple-Mail-63-668738614 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGHjCCAtcw ggJAoAMCAQICAw3FWTANBgkqhkiG9w0BAQQFADBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0EwHhcNMDUwMTEwMDA0MTA5WhcNMDYwMTEwMDA0MTA5WjBHMR8wHQYDVQQD ExZUaGF3dGUgRnJlZW1haWwgTWVtYmVyMSQwIgYJKoZIhvcNAQkBFhVDcmFpZy5SdXNzZWxsQFN1 bi5DT00wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDti7ZE4rO6oXKbLM02AG9WY55t udmBVL53fb3V3X5S1kvcJOk1NEMIYT/T7Ww+/hE955zvHT29+mIoNe8AW/yj1WUH8uGG2HxhwCHI UQTHmN/ioVJgjwUaYbtNMKbL/NRpnL0QWewdMJS+6IFzFyX7ADFW5cJ+UWNLvNeWAQtN0mtLildn vdOgh50i8YPvACNkCHoomGjXx0azcXbe1X3c5AgRI6e2CZe5k2lRFQFUMqkjdoMtQPoNqJ1BxH9l i4cnabl8mcTwHHl44hrvb8ThqwRf2pfJh2vVuwmgK6z4IWjOk4RQM+0ODsRdq67mBdimJYmPMK1p RMBHzrUsfKxNAgMBAAGjMjAwMCAGA1UdEQQZMBeBFUNyYWlnLlJ1c3NlbGxAU3VuLkNPTTAMBgNV HRMBAf8EAjAAMA0GCSqGSIb3DQEBBAUAA4GBAIj86LzxCHedStDLMEeqHLy+UFG7zIRHfChSIV42 +MvXicydXEBh8v0Ry1V2d/lY4jS78G5yW5R9fKt1U5nlRBCOVzdhomvSolnNRIT71wPVVDrAIVlA YpXKxSmVBq7+4hV+3ZLHDeq3qZnNmiJR0sTEUD16xZX1RJs9dRYPCHoRMIIDPzCCAqigAwIBAgIB DTANBgkqhkiG9w0BAQUFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTES MBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMf Q2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFs IEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29t MB4XDTAzMDcxNzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBG cmVlbWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/cVbL rzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8YQRAHmQZ cmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4oDagNIYyaHR0 cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5jcmwwCwYDVR0PBAQD AgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwyLTEzODANBgkqhkiG9w0B AQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4Nh+qLZ82L8D0HFAgk3A8/a3h YWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowgT2Vfldr394fWxghOrvbqNOUQGls1 TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCAucwggLjAgEBMGkwYjELMAkGA1UEBhMCWkExJTAjBgNV BAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h bCBGcmVlbWFpbCBJc3N1aW5nIENBAgMNxVkwCQYFKw4DAhoFAKCCAVMwGAYJKoZIhvcNAQkDMQsG CSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDUwOTE5MTc1MTUwWjAjBgkqhkiG9w0BCQQxFgQU qIeOaDe+LrbwWQl83JzT/T54WiUweAYJKwYBBAGCNxAEMWswaTBiMQswCQYDVQQGEwJaQTElMCMG A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECAw3FWTB6BgsqhkiG9w0BCRACCzFroGkwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1Ro YXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAgMNxVkwDQYJKoZIhvcNAQEBBQAEggEA a2QCgX9UCJowOrCLqcTDv5x4U5VRoDu5ewBBhdYSXhGfSCdpkWYYgc41ncxA5K2zkvglgAwSuGxy HNGiDt1nswKD8vQMCYb/GwOeuBmsgSSvbRftc6q50Dpieg9RxDU/igzgrZuPLwg8ZWp4Iaqn/vIq 6D2XokOsLp9wWIqVkYjHRPfUp8CWHPXqhHgPC6CGv3UCSYQUmfdyxV9OHSmPu2WMo3dj2Y4e17Pz 3GQVAGL6ssMuoxYgFYgzlje7nDymeIXJcIzjwmXWDU7DSoH3GkyMlvMzveFd1hp/LyI6fDTjBUwG MqJc7t8zMLFg1gr+ayMadHNn0wY6fx5vGmqoqAAAAAAAAA== --Apple-Mail-63-668738614--