Return-Path: X-Original-To: apmail-openjpa-commits-archive@www.apache.org Delivered-To: apmail-openjpa-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6587262F6 for ; Thu, 4 Aug 2011 17:42:10 +0000 (UTC) Received: (qmail 87264 invoked by uid 500); 4 Aug 2011 17:42:10 -0000 Delivered-To: apmail-openjpa-commits-archive@openjpa.apache.org Received: (qmail 87205 invoked by uid 500); 4 Aug 2011 17:42:09 -0000 Mailing-List: contact commits-help@openjpa.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openjpa.apache.org Delivered-To: mailing list commits@openjpa.apache.org Received: (qmail 87198 invoked by uid 99); 4 Aug 2011 17:42:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Aug 2011 17:42:09 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Aug 2011 17:42:07 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id EA86123889B1 for ; Thu, 4 Aug 2011 17:41:46 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1153950 - in /openjpa/branches/2.0.x: openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/kernel/ openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/ openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/generationtype/ ... Date: Thu, 04 Aug 2011 17:41:46 -0000 To: commits@openjpa.apache.org From: hthomann@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20110804174146.EA86123889B1@eris.apache.org> Author: hthomann Date: Thu Aug 4 17:41:44 2011 New Revision: 1153950 URL: http://svn.apache.org/viewvc?rev=1153950&view=rev Log: OPENJPA-1376: Backported trunk changes to 2.0.x where possible, made code gated by a system property. Modified: openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/kernel/NativeJDBCSeq.java openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DB2Dictionary.java openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DBDictionary.java openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/OracleDictionary.java openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/PostgresDictionary.java openjpa/branches/2.0.x/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/generationtype/TestNativeSeqGenerator.java openjpa/branches/2.0.x/openjpa-project/src/doc/manual/jpa_overview_mapping.xml openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_dbsetup.xml openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_optimization.xml openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_runtime.xml Modified: openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/kernel/NativeJDBCSeq.java URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/kernel/NativeJDBCSeq.java?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/kernel/NativeJDBCSeq.java (original) +++ openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/kernel/NativeJDBCSeq.java Thu Aug 4 17:41:44 2011 @@ -55,6 +55,8 @@ import serp.util.Strings; /** * {@link JDBCSeq} implementation that uses a database sequences * to generate numbers. + * Supports allocation (caching). In order for allocation to work properly, the database sequence must be defined + * with INCREMENT BY value equal to allocate * increment. * * @see JDBCSeq * @see AbstractJDBCSeq @@ -73,9 +75,11 @@ public class NativeJDBCSeq private DBIdentifier _seqName = DBIdentifier.newSequence("OPENJPA_SEQUENCE"); private int _increment = 1; private int _initial = 1; - private int _allocate = 0; + private int _allocate = 50; private Sequence _seq = null; private String _select = null; + private long _nextValue = 0; + private long _maxValue = -1; // for deprecated auto-configuration support private String _format = null; @@ -203,29 +207,76 @@ public class NativeJDBCSeq public void endConfiguration() { buildSequence(); - DBDictionary dict = _conf.getDBDictionaryInstance(); - if (_format == null) { - _format = dict.nextSequenceQuery; - if (_format == null) - throw new MetaDataException(_loc.get("no-seq-sql", _seqName)); - } if (DBIdentifier.isNull(_tableName)) _tableName = DBIdentifier.newTable("DUAL"); + DBDictionary dict = _conf.getDBDictionaryInstance(); String name = dict.getFullName(_seq); - Object[] subs = (_subTable) ? new Object[]{ name, _tableName } + + if (dict.useNativeSequenceCache){ + if (_format == null) { + _format = dict.nextSequenceQuery; + if (_format == null) + throw new MetaDataException(_loc.get("no-seq-sql", _seqName)); + } + + Object[] subs = (_subTable) ? new Object[]{ name, _tableName } : new Object[]{ name }; - _select = MessageFormat.format(_format, subs); + _select = MessageFormat.format(_format, subs); + } + else { + String format = dict.nextSequenceQuery; + if (format == null) { + throw new MetaDataException(_loc.get("no-seq-sql", _seqName)); + } + + // Increment step is needed for Firebird which uses non-standard sequence fetch syntax. + // Use String.valueOf to get rid of possible locale-specific number formatting. + _select = MessageFormat.format(format, new Object[]{name, + String.valueOf(_allocate * _increment)}); + } type = dict.nativeSequenceType; } @Override - protected Object nextInternal(JDBCStore store, ClassMapping mapping) + protected synchronized Object nextInternal(JDBCStore store, ClassMapping mapping) + throws SQLException { + DBDictionary dict = _conf.getDBDictionaryInstance(); + + //To maintain existing behavior call allocateInternal to get the next + //sequence value, which it stores in _nextValue, and simply return the value. + if (dict.useNativeSequenceCache){ + allocateInternal(0, store, mapping); + return _nextValue; + } + + if (_nextValue < _maxValue) { + long result = _nextValue; + _nextValue += _increment; + return result; + } + + allocateInternal(0, store, mapping); + long result = _nextValue; + _nextValue += _increment; + return result; + } + + /** + * Allocate additional sequence values. + * @param additional ignored - the allocation size is fixed and determined by allocate and increment properties. + * @param store used to obtain connection + * @param mapping ignored + */ + @Override + protected synchronized void allocateInternal(int additional, JDBCStore store, ClassMapping mapping) throws SQLException { + Connection conn = getConnection(store); try { - return getSequence(conn); + _nextValue = getSequence(conn); + _maxValue = _nextValue + _allocate * _increment; } finally { closeConnection(conn); } @@ -300,9 +351,7 @@ public class NativeJDBCSeq try { stmnt = conn.prepareStatement(_select); dict.setTimeouts(stmnt, _conf, false); - synchronized(this) { - rs = stmnt.executeQuery(); - } + rs = stmnt.executeQuery(); if (rs.next()) return rs.getLong(1); @@ -327,13 +376,12 @@ public class NativeJDBCSeq * Where the following options are recognized. *
    *
  • -properties/-p <properties file or resource>: The - * path or resource name of a OpenJPA properties file containing - * information such as the license key and connection data as + * path or resource name of an OpenJPA properties file containing + * information such as connection data as * outlined in {@link JDBCConfiguration}. Optional.
    • *
  • -<property name> <property value>: All bean * properties of the OpenJPA {@link JDBCConfiguration} can be set by - * using their names and supplying a value. For example: - * -licenseKey adslfja83r3lkadf
    • + * using their names and supplying a value. *
* The various actions are as follows. *
    @@ -373,7 +421,7 @@ public class NativeJDBCSeq } /** - * Run the tool. Return false if an invalid option was given. + * Run the tool. Returns false if an invalid option was given. */ public static boolean run(JDBCConfiguration conf, String[] args, String action) Modified: openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DB2Dictionary.java URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DB2Dictionary.java?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DB2Dictionary.java (original) +++ openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DB2Dictionary.java Thu Aug 4 17:41:44 2011 @@ -227,12 +227,15 @@ public class DB2Dictionary } } - public String[] getCreateSequenceSQL(Sequence seq) { - String[] sql = super.getCreateSequenceSQL(seq); - if (seq.getAllocate() > 1) - sql[0] += " CACHE " + seq.getAllocate(); - return sql; - } + public String[] getCreateSequenceSQL(Sequence seq) { + String[] sql = super.getCreateSequenceSQL(seq); + + if (seq.getAllocate() > 1 && useNativeSequenceCache){ + sql[0] += " CACHE " + seq.getAllocate(); + } + + return sql; + } @Override protected String getSequencesSQL(String schemaName, String sequenceName) { Modified: openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DBDictionary.java URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DBDictionary.java?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DBDictionary.java (original) +++ openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/DBDictionary.java Thu Aug 4 17:41:44 2011 @@ -350,6 +350,7 @@ public class DBDictionary public String sequenceNameSQL = null; // most native sequences can be run inside the business transaction public int nativeSequenceType= Seq.TYPE_CONTIGUOUS; + public boolean useNativeSequenceCache = true; protected JDBCConfiguration conf = null; protected Log log = null; @@ -3385,6 +3386,14 @@ public class DBDictionary buf.append(seqName); if (seq.getInitialValue() != 0) buf.append(" START WITH ").append(seq.getInitialValue()); + + if (seq.getIncrement() > 1 && useNativeSequenceCache){ + buf.append(" INCREMENT BY ").append(seq.getIncrement()); + } + else if ((seq.getIncrement() > 1) || (seq.getAllocate() > 1)){ + buf.append(" INCREMENT BY ").append(seq.getIncrement() * seq.getAllocate()); + } + if (seq.getIncrement() > 1) buf.append(" INCREMENT BY ").append(seq.getIncrement()); return new String[]{ buf.toString() }; Modified: openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/OracleDictionary.java URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/OracleDictionary.java?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/OracleDictionary.java (original) +++ openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/OracleDictionary.java Thu Aug 4 17:41:44 2011 @@ -1036,12 +1036,15 @@ public class OracleDictionary } @Override - public String[] getCreateSequenceSQL(Sequence seq) { - String[] sql = super.getCreateSequenceSQL(seq); - if (seq.getAllocate() > 1) - sql[0] += " CACHE " + seq.getAllocate(); - return sql; - } + public String[] getCreateSequenceSQL(Sequence seq) { + String[] sql = super.getCreateSequenceSQL(seq); + + if (seq.getAllocate() > 1 && useNativeSequenceCache){ + sql[0] += " CACHE " + seq.getAllocate(); + } + + return sql; + } /** * Return the preferred {@link Types} constant for the given Modified: openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/PostgresDictionary.java URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/PostgresDictionary.java?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/PostgresDictionary.java (original) +++ openjpa/branches/2.0.x/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/sql/PostgresDictionary.java Thu Aug 4 17:41:44 2011 @@ -345,8 +345,11 @@ public class PostgresDictionary @Override public String[] getCreateSequenceSQL(Sequence seq) { String[] sql = super.getCreateSequenceSQL(seq); - if (seq.getAllocate() > 1) + + if (seq.getAllocate() > 1 && useNativeSequenceCache){ sql[0] += " CACHE " + seq.getAllocate(); + } + return sql; } Modified: openjpa/branches/2.0.x/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/generationtype/TestNativeSeqGenerator.java URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/generationtype/TestNativeSeqGenerator.java?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/generationtype/TestNativeSeqGenerator.java (original) +++ openjpa/branches/2.0.x/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/generationtype/TestNativeSeqGenerator.java Thu Aug 4 17:41:44 2011 @@ -59,7 +59,38 @@ public class TestNativeSeqGenerator exte em.getTransaction().commit(); int genId = entityE2.getId(); int nextId = (int)((Long)em.getIdGenerator(EntityE2.class).next()).longValue(); - assertTrue("Next value should depend on previous genid", nextId == genId + 1); + assertTrue("Next value should depend on previous genid", nextId >= genId + 1); em.close(); } + + /** + * Asserts native sequence generator allocates values in memory + * and requests sequence values from database only when necessary. + */ + public void testAllocationSize() { + //Run this test only if the user has elected to not use the Native Sequence Cache. + if (supportsNativeSequence && !dict.useNativeSequenceCache){ + // Turn off statement batching for easier INSERT counting. + dict.setBatchLimit(0); + em.getTransaction().begin(); + resetSQL(); + for (int i = 0; i < 51; i++) { + createEntityE2(); + em.persist(entityE2); + } + em.getTransaction().commit(); + + // Since allocationSize has a default of 50, we expect 2 sequence fetches and 51 INSERTs. + assertEquals("53 statements should be executed.", 53, getSQLCount()); + String[] statements = new String[53]; + statements[0] = ".*"; + statements[1] = ".*"; + for (int i = 2; i < 53; i++) { + statements[i] = "INSERT .*"; + } + assertAllExactSQLInOrder(statements); + em.close(); + } + } + } Modified: openjpa/branches/2.0.x/openjpa-project/src/doc/manual/jpa_overview_mapping.xml URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-project/src/doc/manual/jpa_overview_mapping.xml?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-project/src/doc/manual/jpa_overview_mapping.xml (original) +++ openjpa/branches/2.0.x/openjpa-project/src/doc/manual/jpa_overview_mapping.xml Thu Aug 4 17:41:44 2011 @@ -967,10 +967,11 @@ default. allocationSize property -int allocationSize: Some databases can pre-allocate groups -of sequence values. This allows the database to service sequence requests from -cache, rather than physically incrementing the sequence with every request. This -allocation size defaults to 50. +int allocationSize: The number of values to allocate in +memory for each trip to the database. Allocating values in memory allows the JPA +runtime to avoid accessing the database for every sequence request. +This number also specifies the amount that the sequence value is incremented +each time the sequence is accessed. Defaults to 50. Modified: openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_dbsetup.xml URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_dbsetup.xml?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_dbsetup.xml (original) +++ openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_dbsetup.xml Thu Aug 4 17:41:44 2011 @@ -1824,8 +1824,9 @@ create a generated name. The default val NextSequenceQuery: A SQL string for obtaining a native sequence value. May use a placeholder of {0} for the variable -sequence name. Defaults to a database-appropriate value. For example, -"SELECT {0}.NEXTVAL FROM DUAL" for Oracle. +sequence name and {1} for sequence increment. +Defaults to a database-appropriate value. For example, +"SELECT {0}.NEXTVAL FROM DUAL" for Oracle database. Modified: openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_optimization.xml URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_optimization.xml?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_optimization.xml (original) +++ openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_optimization.xml Thu Aug 4 17:41:44 2011 @@ -295,8 +295,7 @@ it can become a factor. For applications that perform large bulk inserts, the retrieval of sequence -numbers can be a bottleneck. Increasing sequence increments and using -table-based rather than native database sequences can reduce or eliminate +numbers can be a bottleneck. Increasing sequence allocation sizes can reduce or eliminate this bottleneck. In some cases, implementing your own sequence factory can further optimize sequence number retrieval. Modified: openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_runtime.xml URL: http://svn.apache.org/viewvc/openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_runtime.xml?rev=1153950&r1=1153949&r2=1153950&view=diff ============================================================================== --- openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_runtime.xml (original) +++ openjpa/branches/2.0.x/openjpa-project/src/doc/manual/ref_guide_runtime.xml Thu Aug 4 17:41:44 2011 @@ -1661,8 +1661,11 @@ properties: -Allocate: Some database can allocate values in-memory to -service subsequent sequence requests faster. +Allocate: The number of values to allocate on each database +trip. Defaults to 50, meaning the class will set aside the next 50 numbers each +time it accesses the sequence, which in turn means it only has to make a +database trip to get new sequence numbers once every 50 sequence number +requests.