openjpa-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Persson <mailto.wo...@gmail.com>
Subject Re: Stress tests and configuration
Date Wed, 09 May 2012 18:22:44 GMT
Hi again.

Just wanted to add that we have reduced the settings to this

 <property name="openjpa.DataCache" value="true(CacheSize=1000,
SoftReferenceSize=0)"/>
 <property name="openjpa.QueryCache" value="false"/>
 <property name="openjpa.Log" value="DefaultLevel=INFO, Runtime=INFO,
Tool=INFO, SQL=INFO"/>
 <property name="openjpa.DetachState"
value="fetch-groups(DetachedStateField=true)"/>



Because we have alot of threads doing database calls we have one function
to handle the fetching of an EntityManager. Threads are both coming from
Servlet calls and stateful worker threads so we need some sort of
separation so we don't get transaction locks.
We solved it with ThreadLocal HashTable with the managers and an static
Hashtable for the factories. This should give us one factory per database
and one entity manager per thread. And because we don't have persistent
data connections we have to check if they are open as well when getting the
connection.


   private static transient ThreadLocal<Hashtable<String, EntityManager>>
entityManagers = new ThreadLocal<Hashtable<String, EntityManager>>();
   private static Hashtable<String, EntityManagerFactory> factories = new
Hashtable<String, EntityManagerFactory>();

   public static EntityManager getEntityManager(String entityName) throws
Exception {
      if(entityManagers.get() == null) {
         entityManagers.set(new Hashtable<String, EntityManager>());
      }

      EntityManager entityManager = entityManagers.get().get(entityName);

      if(entityManager != null) {
         OpenJPAEntityManager oem = OpenJPAPersistence.cast(entityManager);
         Connection con = (Connection) oem.getConnection();
         if(con.isClosed()) {
            entityManager = null;
         }
      }

      if(entityManager == null || !entityManager.isOpen()) {
         if(factories.get(entityName) == null) {
            factories.put(entityName,
Persistence.createEntityManagerFactory(entityName, System.getProperties()));
         }

         entityManager =
factories.get(entityName).createEntityManager()
         entityManagers.get().put(entityName, entityManager);
      }
      return entityManager;
   }

We work hard on finding a solution and getting a stable system so any
assistance is very appreciated.

Best regards

Daniel


On Wed, May 9, 2012 at 3:44 PM, Daniel Persson <mailto.woden@gmail.com>wrote:

> Hi Rick.
>
> At the moment we have a problem with EntityManagers not being in sync. We
> have an application where you could update user data. And when we reload
> the page with the user data listed. These are elements connected to the
> user data and are them self another entity entirely and we fetch them with
> "select e from entity e where e.userid = :gaUserId".
> When removing or adding an entity we could reload the page and depending
> on which entitymanager deliver the result we see the added or removed
> entities. So the result set size could differ from 1,2,3,4 with 4 reloads
> of the web browser. So we need consistancy between EntityManagers.
>
> The data saved to database is in sync but the managers in each thread have
> different result sets saved.
>
> I haven't been able to reproduce the problem with TRACE logging. The error
> occurs from time to time in my simple example but it occurred when I tested
> without trace.
> I found another error though with trace log.
>
> Caused by: <openjpa-2.1.1-r422266:1148538 nonfatal general error>
> org.apache.openjpa.persistence.PersistenceException: index = 51
>
>     at
> org.apache.openjpa.kernel.BrokerImpl.afterCompletion(BrokerImpl.java:2018)
>     at
> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:94)
>     at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1498)
>     at
> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:933)
>     at
> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:569)
>     ... 1 more
> Caused by: java.lang.IndexOutOfBoundsException: index = 51
>     at serp.bytecode.lowlevel.ConstantPool.getEntry(ConstantPool.java:52)
>     at
> serp.bytecode.lowlevel.ComplexEntry.getNameAndTypeEntry(ComplexEntry.java:85)
>     at
> serp.bytecode.FieldInstruction.getFieldTypeName(FieldInstruction.java:220)
>     at
> serp.bytecode.GetFieldInstruction.getStackChange(GetFieldInstruction.java:22)
>     at serp.bytecode.Code.calculateMaxStack(Code.java:214)
>     at
> org.apache.openjpa.enhance.DynamicStorageGenerator.addSetMethod(DynamicStorageGenerator.java:342)
>     at
> org.apache.openjpa.enhance.DynamicStorageGenerator.addSetMethods(DynamicStorageGenerator.java:272)
>     at
> org.apache.openjpa.enhance.DynamicStorageGenerator.generateStorage(DynamicStorageGenerator.java:121)
>     at
> org.apache.openjpa.enhance.PCDataGenerator.generateStorage(PCDataGenerator.java:116)
>     at
> org.apache.openjpa.enhance.PCDataGenerator.generatePCData(PCDataGenerator.java:93)
>     at
> org.apache.openjpa.datacache.DataCacheStoreManager.newPCData(DataCacheStoreManager.java:790)
>     at
> org.apache.openjpa.datacache.DataCacheStoreManager.updateCaches(DataCacheStoreManager.java:146)
>     at
> org.apache.openjpa.datacache.DataCacheStoreManager.commit(DataCacheStoreManager.java:89)
>     at
> org.apache.openjpa.kernel.DelegatingStoreManager.commit(DelegatingStoreManager.java:95)
>     at
> org.apache.openjpa.kernel.BrokerImpl.endStoreManagerTransaction(BrokerImpl.java:1455)
>     at
> org.apache.openjpa.kernel.BrokerImpl.endTransaction(BrokerImpl.java:2335)
>     at
> org.apache.openjpa.kernel.BrokerImpl.afterCompletion(BrokerImpl.java:1994)
>     ... 5 more
>
> Best regards
>
> Daniel
>
>
> On Tue, May 8, 2012 at 3:09 PM, Rick Curtis <curtisr7@gmail.com> wrote:
>
>> Daniel -
>>
>> Yes, I'd be interested in seeing the full exception. That being said,
>> you're probably safe just turning off the DynamicDataStructs . In our
>> performance testing, we never observed an improvement when that property
>> is
>> enabled.
>>
>> Since we're on the topic of properties, I'd also recommend getting rid of
>> the openjpa.Multithreaded property. Sharing EntityManagers / Entities
>> across multiple threads isn't something that OpenJPA was designed to do.
>> openjpa.Multithreaded was supposed to address that concern, but it did so
>> poorly. There are a number of outstanding JIRAs that document some of the
>> issues... I can dig them up if you'd like?
>>
>> Thanks,
>> Rick
>>
>> On Tue, May 8, 2012 at 1:43 AM, Daniel Persson <mailto.woden@gmail.com
>> >wrote:
>>
>> > Hi Rick.
>> >
>> > These are the actual stacktraces that you get with log level WARN. I
>> > changed to TRACE and got more information that actually lead me to
>> beleave
>> > that the DynamicDataStructs where the problem. For some reason it could
>> not
>> > handle special types of data. Without DynamicDataStructs it seems to
>> work
>> > at the moment.
>> >
>> > Didn't save the stacktrace but I'm sure I got the null pointer
>> exception on
>> >
>> > 365: BCMethod method = bc.declareMethod(name, type, new Class[]{
>> int.class
>> > });
>> >
>> > in DynamicStorageGenerator.java
>> >
>> > It didn't recognize the type supplied. If your interested I could turn
>> the
>> > feature on and get a trace for you.
>> >
>> > Best regards
>> >
>> > Daniel
>> >
>> > On Mon, May 7, 2012 at 3:59 PM, Rick Curtis <curtisr7@gmail.com> wrote:
>> >
>> > > Daniel -
>> > >
>> > > Can you post the entire stacktrace? You've snipped out the important
>> > parts.
>> > >
>> > > Thanks,
>> > > Rick
>> > >
>> > > On Mon, May 7, 2012 at 3:19 AM, Daniel Persson <
>> mailto.woden@gmail.com
>> > > >wrote:
>> > >
>> > > > Hi.
>> > > >
>> > > > I'd used OpenJPA for my projects for a while now and started to use
>> it
>> > at
>> > > > work.
>> > > >
>> > > > Our service runs on Tomcat and have a lot of users. So after
>> > implementing
>> > > > OpenJPA for a few months we are now in the testing phase.
>> > > > During the stress test adding a few hundred records using different
>> > > tomcat
>> > > > threads I get the same OpenJPA error over and over.
>> > > >
>> > > > I've changed the configuration back and forth to resolve the issue.
>> > > > Sometimes I get it to work for a while and then it breaks again.
>> > > > So I would like to ask what the error could be a result of and if
I
>> > have
>> > > > any obvious configuration faults.
>> > > >
>> > > > I've written a small test class adding entities using 1000 threads
>> and
>> > > this
>> > > > will generate the errors from time to time.
>> > > > The code uses one factory and a lot of managers.
>> > > >
>> > > > <openjpa-2.1.1-r422266:1148538 fatal store error>
>> > > > org.apache.openjpa.persistence.RollbackException: null
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:593)
>> > > >
>> > > > Caused by: <openjpa-2.1.1-r422266:1148538 nonfatal general error>
>> > > > org.apache.openjpa.persistence.PersistenceException: null
>> > > >        at
>> > > >
>> > >
>> >
>> org.apache.openjpa.kernel.BrokerImpl.afterCompletion(BrokerImpl.java:2018)
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:94)
>> > > >        at
>> > > org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1498)
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:933)
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:569)
>> > > >        ... 27 more
>> > > > Caused by: java.lang.NullPointerException
>> > > >
>> > > >
>> > > > <persistence xmlns="http://java.sun.com/xml/ns/persistence"
>> > xmlns:xsi="
>> > > > http://www.w3.org/2001/XMLSchema-instance" version="1.0">
>> > > >    <persistence-unit name="userDbPersistence"
>> > > > transaction-type="RESOURCE_LOCAL">
>> > > >
>> > >  <non-jta-data-source>java:comp/env/jdbc/userDb</non-jta-data-source>
>> > > >
>> > > >        <class>......</class>
>> > > >
>> > > >        <properties>
>> > > >            <property name="openjpa.Multithreaded" value="true"
/>
>> > > >            <property name="openjpa.InverseManager" value="true"
/>
>> > > >            <property name="openjpa.DynamicDataStructs"
>> value="true"/>
>> > > >            <property name="openjpa.Compatibility"
>> > > >
>> > > >
>> > >
>> >
>> value="QuotedNumbersInQueries=true,CopyOnDetach=true,cascadeWithDetach=true,superclassDiscriminatorStrategyByDefault=false"
>> > > > />
>> > > >            <property name="openjpa.DataCache"
>> > value="true(CacheSize=1000,
>> > > > SoftReferenceSize=0)"/>
>> > > >            <property name="openjpa.QueryCache" value="false"/>
>> > > >            <property name="openjpa.RemoteCommitProvider"
>> value="sjvm"/>
>> > > >            <property name="openjpa.Log" value="DefaultLevel=WARN,
>> > > > Runtime=WARN, Tool=WARN, SQL=WARN"/>
>> > > >            <property name="openjpa.FetchBatchSize" value="0"/>
>> > > >            <property name="openjpa.DetachState"
>> > > > value="fetch-groups(DetachedStateField=true)"/>
>> > > >            <property name="openjpa.LockTimeout" value="3000"/>
>> > > >            <property name="openjpa.jdbc.EagerFetchMode"
>> value="join" />
>> > > >            <property name="openjpa.jdbc.SubclassFetchMode"
>> > > > value="parallel" />
>> > > >        </properties>
>> > > >    </persistence-unit>
>> > > > </persistence>
>> > > >
>> > > > Any tips or suggestions are much appreciated. Have tried a lot of
>> > > different
>> > > > configurations. One requirement for the configuration as well is
>> that
>> > we
>> > > > want to use some sort of caching because this was one of the reason
>> for
>> > > the
>> > > > shift to openJPA.
>> > > >
>> > > > Best regards
>> > > >
>> > > > Daniel
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > *Rick Curtis*
>> > >
>> >
>>
>>
>>
>> --
>> *Rick Curtis*
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message