openjpa-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Curtis <curti...@gmail.com>
Subject Re: Bug? DataCache after update contains only the updated information from a Collection
Date Fri, 08 Jun 2012 19:29:19 GMT
John -

It's Friday afternoon, so I haven't really dug into this issue deeply, but
for giggles can you try to set openjpa.InverseManager to false? I struggled
to recreate your failure, but as soon as I set that property to true I
started seeing the same behavior.

Thanks,
Rick

On Fri, Jun 8, 2012 at 10:43 AM, Kevin Sutter <kwsutter@gmail.com> wrote:

> Hi John,
> 1.  The @InverseLogical annotation shouldn't affect your scenario.  I was
> just curious as to why you were using it.  My ignorance of the usefulness
> of this property would shy me away from its usage, but that's just me.
>
> 2.  Thanks for the explanation of your update() method.  This sounds like
> the proper usage of the merge() processing.
>
> 3.  Thanks.  Just curious on your persistence model.
>
> 4.  Your persistence.xml looks fine.  Nothing else jumps out at me.
>
> This leaves us at figuring out a simple testcase to reproduce the problem.
> Since we have many JUnits with caching and collections already, I'm
> surprised that we haven't discovered the issue prior to this.  I'm not sure
> if I'll have time today to create or modify or JUnit to reproduce the issue
> (taking some vacation).  If you have any cycles to simplify your scenario,
> it would help.  Otherwise, maybe early next week we can get to the bottom
> of this...
>
> I did look into your comments concerning the openjpa.RefreshFromDataCache
> property...  Unfortunately, this property doesn't seem to be working as you
> might expect.  First, it's not documented in our OpenJPA manual.  And, when
> I look at the code, it doesn't look like the value of this property is
> being used properly to bypass the cache.  So, I wouldn't count on this.
>
> I then looked at the proper means of specifying a datacache bypass with the
> javax.persistence.cache.RetrieveMode properties as defined by JPA 2.0.  You
> can specify a value of USE or BYPASS that can be used for finds, queries,
> and refresh operations.  But, then there's this statement in the spec:
>
> *"The retrieveMode property is ignored for the refresh method,*
> *which always causes data to be retrieved from the database, not the
> cache."
> *
>
> You have mentioned that you are performing a refresh.  Are you calling
> em.refresh(object)?  If so, this sounds like there might be a bug in that
> processing...  You could try passing in the RetrieveMode property of BYPASS
> on the refresh just to see if it makes a difference.
>
> That's all I got right now...  Have a good weekend!
>
> Kevin
>
> On Fri, Jun 8, 2012 at 1:57 AM, Boblitz John <John.Boblitz@bertschi.com
> >wrote:
>
> > Hi Kevin,
> >
> > Thanks for the response.  Here some answers:
> >
> > 1. "Why is the @InverseLogical annotation required?"
> > I included this quite early in the development of  as the manual
> suggested
> > that
> > using the annotation would cause openJpa to manage the relations for me.
> > Are you implying that mappedBy in uneeded if I use @InverseLogical?
> > I could remove the annotation and check - but this is prevalent
> > throuhout the model, and I have not noticed any problems elsewhere (yet).
> >
> > 2.  "Can you explain the use of the update() method?"
> > update receives the detached entities which are then merged and
> committed.
> > The application is on three tiers in a Java SE Environment- DB -> App ->
> > Gui.  The GUI
> > gets the data in one step and the enitities are detached.  At some point
> > later, the data is
> > passed back to the app.  The process is, basically:
> >
> > Get a manager
> > Read the data from a DTO Object
> > Validate the data
> > Create entity object and fill with DTO Data
> > Merge into the context
> > Commit
> >
> > I am stuck with using separate DTO Objects which unfortunately also break
> > the references as only the
> > uid is passed - but, other than the overhead, I have noticed no problems
> > in doing so
> >
> > 3.  "How is the lifecycle of the EM being managed, ...?"
> > As I mentioned in 2, since I'm in SE, each request essentially gets it's
> > own manager.
> >
> > 4.  "Can you post your whole persistence.xml? "
> > Below my persistence.xml.  Yes, the QueryCache & QuerySQLCache are
> active.
> >
> > I just reran the scenario and produced a trace (some very nasty sqls in
> > there as the apps are
> > all still using default fetch plans and the Equipment entity is complex
> :D
> > )
> >
> > Essentially, when the merge() is executed - several selects are executed
> > and since
> > nothing changed yet, all info is pulled from cache.
> >
> > Then, several updates / inserts are executed as expected.  Just after
> > these statements I get another
> > list of hits on the cache ...
> >
> > Then I get this: (413938 is the Equipment Entity and 502801 is the new
> > Axle]
> >
> > - Performing a commit on the cache. Adding [502801], updating [] and
> > [414690, 414571, 414570, 414082, 414688, 414689, 414626, 414627, 414624,
> > 413938, 414572, 414625, 414681,
> > 414621, 414680, 414623, 414683, 414622, 414682, 414685, 414684, 414687,
> > 414686, 414679], and removing [].
> >
> > Just after that, I do a refresh and this shows:
> > - Cache hit while looking up key "413938".
> >
> > Here I expected that the cache would not be used and that the DB would be
> > read to resynch the cache since
> > I have:  openjpa.RefreshFromDataCache: false
> >
> >
> >
> > I appreciate the help!!!
> >
> > 
> >
> > John
> >
> > ----
> >
> > Who is General Failure, and why is he reading my hard disk?
> >
> >
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <persistence
> >        version="2.0"
> >        xmlns="http://java.sun.com/xml/ns/persistence"
> >        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >        xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
> >
> > http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
> > >
> >
> >        <persistence-unit
> >                name="g11.persistence"
> >                transaction-type="RESOURCE_LOCAL"
> >        >
> >
> >
> >  <provider>org.apache.openjpa.persistence.PersistenceProviderImpl
> > </provider>
> >
> >
> >                <!-- JPQL Named Queries -->
> >                <mapping-file>META-INF/dbNamedQueries.xml</mapping-file>
> > <!-- JPQL Named Queries generated from the database schema.xml -->
> >                <mapping-file>META-INF/guiNamedQueries.xml</mapping-file>
> >  <!-- JPQL Named Queries generated for gui selection screens -->
> >                <mapping-file>META-INF/acNamedQueries.xml</mapping-file>
> > <!-- JPQL Named Queries generated for gui selection screens -->
> >                <mapping-file>META-INF/bdisNamedQueries.xml</mapping-file>
> > <!-- JPQL Named Queries generated for general bdis queries -->
> >
> >
> >
> >                <!-- Class Definitions -->
> >                <class> base.BaseEntity</class>
> >                <class> ams.locations.BusinessPartner</class>
> >                <class> ams.locations.Country</class>
> >                <class> ams.locations.CountryGroup</class>
> >                <class> ams.locations.Location</class>
> >                <class> ams.locations.Subdivision</class>
> >                <class> ams.locations.ValueAddedTax</class>
> >                <class> com.codes.Currency</class>
> >                <class> com.codes.ExchangeRate</class>
> >                <class> com.codes.ModeOfTransportation</class>
> >                <class> com.codes.Notes</class>
> >                <class> com.codes.Period</class>
> >                <class> com.codes.Text</class>
> >                <class> com.codes.TranslatedText</class>
> >                <class> com.codes.UnitOfMeasurement</class>
> >                <class> com.codes.WritingSystem</class>
> >                <class> com.company.BusinessUnit</class>
> >                <class> com.company.Company</class>
> >                <class> com.company.Department</class>
> >                <class> com.company.OrganisationalUnit</class>
> >                <class> com.company.ServiceCenter</class>
> >                <class> com.system.Printer</class>
> >                <class> com.system.UserGroup</class>
> >                <class> com.system.Users</class>
> >                <class> com.system.Permission</class>
> >                <class> com.system.EntityExport</class>
> >                <class> fms.equipment.Equipment</class>
> >                <class> fms.equipment.EquipmentAttachment</class>
> >                <class> fms.equipment.EquipmentNote</class>
> >                <class> fms.equipment.HireOut</class>
> >                <class> fms.equipment.RentalAgreement</class>
> >                <class> fms.equipment.TransferAgreement</class>
> >                <class> fms.equipment.certifications.Certification</class>
> >                <class>
> > fms.equipment.certifications.CertificationType</class>
> >                <class>
> > fms.equipment.certifications.DriverCertification</class>
> >                <class>
> > fms.equipment.certifications.DriverCertificationDocument</class>
> >                <class>
> > fms.equipment.certifications.DriverRequiredCertifications</class>
> >                <class>
> > fms.equipment.certifications.EquipmentCertification</class>
> >                <class>
> > fms.equipment.certifications.EquipmentCertificationDocument</class>
> >                <class>
> > fms.equipment.certifications.EquipmentRequiredCertifications</class>
> >                <class> fms.equipment.certifications.Issuer</class>
> >                <class> fms.equipment.components.Axle</class>
> >                <class> fms.equipment.components.Chamber</class>
> >                <class> fms.equipment.components.Chassis</class>
> >                <class>
> fms.equipment.components.TankBoxUsageHistory</class>
> >                <class> fms.equipment.components.Hose</class>
> >                <class> fms.equipment.components.Motor</class>
> >                <class> fms.equipment.components.MotorUsage</class>
> >                <class> fms.equipment.components.TankBox</class>
> >                <class> fms.equipment.damage.DamageHow</class>
> >                <class> fms.equipment.damage.DamageWhat</class>
> >                <class> fms.equipment.damage.DamageWhere</class>
> >                <class> fms.equipment.damage.EquipmentDamage</class>
> >                <class> fms.equipment.properties.Characteristic</class>
> >                <class>
> fms.equipment.properties.CharacteristicType</class>
> >                <class> fms.equipment.properties.EquipmentType</class>
> >                <class> fms.equipment.properties.Manufacturer</class>
> >                <class> fms.equipment.properties.Model</class>
> >                <class> fms.equipment.properties.PropertyType</class>
> >                <class> fms.equipment.properties.Specification</class>
> >                <class> fms.equipment.properties.SpecificationType</class>
> >                <class>
> > fms.equipment.properties.TechnicalCharacteristic</class>
> >                <class> fms.equipment.service.Service</class>
> >                <class> fms.equipment.service.ServiceHistory</class>
> >                <class> fms.equipment.service.ServiceType</class>
> >                <class> fms.equipment.service.ServicesOffered</class>
> >                <class> hrs.Driver</class>
> >                <class> hrs.DriverRestriction</class>
> >                <class> hrs.DriverVacationModel</class>
> >                <class> hrs.Employee</class>
> >                <class> hrs.RestrictionType</class>
> >
> >
> >                <properties>
> >
> >                        <!-- Logging -->
> >                        <property
> >                                name="openjpa.Log"
> >                                value="log4j" />
> >
> >                        <!-- Connection -->
> >                        <property
> >                                name="openjpa.ConnectionDriverName"
> >                                value="org.postgresql.Driver" />
> >                        <property
> >                                name="openjpa.ConnectionProperties"
> >                                value="MaxActive=100, MaxIdle=5,
> MinIdle=2,
> > MaxWait=60000" />
> >                        <property
> >                                name="openjpa.ConnectionFactoryProperties"
> >                                value="QueryTimeOut=5000,
> PrettyPrint=true,
> > PrettyPrintLineLength=80, PrintParameters=true" />
> >
> >                        <property
> >                                name="openjpa.ConnectionURL"
> >                                value=**** />
> >                        <property
> >                                name="openjpa.ConnectionUserName"
> >                                value=**** />
> >                        <property
> >                                name="openjpa.ConnectionPassword"
> >                                value=**** />
> >                        <property
> >                                name="openjpa.jdbc.DBDictionary"
> >
> >  value="postgres(supportsNullTableForGetImportedKeys=false)" />
> >
> >                        <!-- Caching -->
> >                         <property
> >                                name="openjpa.DataCache"
> >                                value="true(CacheSize=5000,
> > EnableStatistics=true)" />
> >                        <property
> >                                name="openjpa.RemoteCommitProvider"
> >                                value="sjvm" />
> >                        <property
> >                                 name="openjpa.QueryCache"
> >                                value="true(CacheSize=1000,
> > SoftReferenceSize=100, EvictPolicy='timestamp')" />
> >                        <property
> >                                name="openjpa.QueryCompilationCache"
> >                                value="all" />
> >                        <property
> >                                name="openjpa.jdbc.QuerySQLCache"
> >                                value="true(EnableStatistics=true)" />
> >
> >
> >                        <!-- Misc -->
> >                        <property
> >                                name="openjpa.InverseManager"
> >                                value="true" />
> >                         <property
> >                                name="openjpa.DetachState"
> >
> >  value="fetch-groups(DetachedStateField=true)" />
> >                         <property
> >                                name="openjpa.jdbc.SchemaFactory"
> >                                value="native(ForeignKeys=true)" />
> >
> >                </properties>
> >        </persistence-unit>
> > </persistence>
> >
> >
> >
> >
> >
> > > -----Ursprüngliche Nachricht-----
> > > Von: Kevin Sutter [mailto:kwsutter@gmail.com]
> > > Gesendet: Donnerstag, 7. Juni 2012 20:48
> > > An: users@openjpa.apache.org
> > > Betreff: Re: Bug? DataCache after update contains only the
> > > updated information from a Collection
> > >
> > > Hi John,
> > > Good background information.  Thanks for the detail.
> > > Unfortunately, nothing is jumping out at me as an "easy fix".
> > >  I do have a couple of observations and/or questions...
> > >
> > > o  Why is the @InverseLogical annotation required?  It would
> > > seem that you have already defined the bidirectional
> > > relationship via the mappedBy attribute on the @OneToMany.
> > > Is there something else you were trying to model by using the
> > > @InverseLogical?  And, does the removal of that annotation
> > > change the processing in any way?
> > >
> > > o  Can you explain the use of the update() method?  Are the
> > > entities being passed into this method detached from a
> > > persistence context?  That's where the merge() method comes
> > > into play -- merging detached entities into a persistence
> > > context.  The merge() method is not used just because there
> > > are updates to an Entity.
> > >
> > > o  Your code example doesn't show the actual interaction with
> > > the EM.  How is the lifecycle of the EM being managed, and is
> > > the same EM being used for all of the interactions with
> > > transactions, persisting, merging, etc?
> > >
> > > o  Can you post your whole persistence.xml?  For example, you
> > > didn't mention the use of the QueryCache property, but it
> > > looks like this is enabled via the dump of the properties.
> > > The QueryCache is no longer automatically enabled when the
> > > DataCache is enabled, so it must be explicitly enabled.  Not
> > > that it should affect this particular scenario, but I'm just
> > > curious if there are other property "mismatches" that I didn't catch.
> > >
> > > o  If none of these questions provide any fruit, then you
> > > might have found a bug and a JIRA will be required.  But,
> > > let's do a bit more Q&A first...
> > >
> > > Thanks,
> > > Kevin
> > >
> > > On Thu, Jun 7, 2012 at 2:05 AM, Boblitz John
> > > <John.Boblitz@bertschi.com>wrote:
> > >
> > > > Hello,
> > > >
> > > > I'm running openJpa 2.2.0 and have a problem with the
> > > DataCache function.
> > > > I spent the better part of two days localizing the problem
> > > and looking
> > > > for solutions, but alas ...
> > > >
> > > > So, hoping some guru out there can spot my error, here the details:
> > > >
> > > > Environment JavaSE
> > > >
> > > > Persistence XML:
> > > > <property name="openjpa.DataCache" value="true(CacheSize=5000,
> > > > EnableStatistics=true)" /> <property
> > > > name="openjpa.RemoteCommitProvider" value="sjvm" /> <property
> > > > name="openjpa.DetachState"
> > > > value="fetch-groups(DetachedStateField=true)" />
> > > >
> > > > Entities:
> > > >
> > > > @MappedSuperclass
> > > > @EntityListeners({ EntityManipulationLogger.class,
> > > EntityLogger.class
> > > > }) public abstract class BaseEntity { @Version
> > > > @Column(columnDefinition = "int8") private long versionId;
> > > >
> > > > @Id
> > > > @GeneratedValue(strategy = GenerationType.SEQUENCE) @Column(name =
> > > > "uniqueid", columnDefinition = "int8") protected long uniqueId;
> > > >
> > > > ...
> > > >
> > > > @Entity
> > > > @Table(name = "Equipment")
> > > > public class Equipment extends BaseEntity {
> > > >
> > > > @JsonManagedReference
> > > > @OneToMany(cascade = CascadeType.ALL, mappedBy = "equipmentId")
> > > > @InverseLogical("equipmentId")
> > > > private Set<Axle> axles = new HashSet<Axle>();
> > > >
> > > > ...
> > > >
> > > > @Entity
> > > > @Table(name = "Axle")
> > > > public class Axle extends BaseEntity {
> > > >
> > > > @JsonBackReference
> > > > @ManyToOne(fetch = FetchType.EAGER)
> > > > @JoinColumn(name = "equipmentId", columnDefinition =
> > > "int8", nullable
> > > > =
> > > > false)
> > > > private Equipment equipmentId;
> > > >
> > > >
> > > > Code used to Update the DataBase:
> > > >
> > > > @Override
> > > > public <T extends BaseEntity> T update(T pEntity) {
> > > >    this.lock.lock();
> > > >   T object = null;
> > > >    try {
> > > >         try {
> > > >                getTransaction().begin();
> > > >                object = merge(pEntity);
> > > >        } catch (Exception e) {
> > > >                mTrc.error("Error in update(): ", e);
> > > >                getTransaction().setRollbackOnly();
> > > >                handleException(e);
> > > >        } finally {
> > > >                if (getTransaction().isActive()) {
> > > >                    if (getTransaction().getRollbackOnly()) {
> > > >                    getTransaction().rollback();
> > > >                } else {
> > > >                    getTransaction().commit();
> > > >                }
> > > >            }
> > > >        }
> > > >    } catch (Exception e) {
> > > >        mTrc.error("Error in update(): ", e);
> > > >        handleException(e);
> > > >    } finally {
> > > >        this.lock.unlock();
> > > >    }
> > > >    return object;
> > > > }
> > > >
> > > > Symptom:
> > > > When updating the Equipment entity, if an additional Axle
> > > is added to
> > > > the Set and the other Axles remain unchanged:
> > > > 1.  Upon entry into update(), pEntity contains all the Axels (new &
> > > > old) including UID & Version.
> > > > 2.  object = merge(pEntity) - performs as expected, object contains
> > > > all the Axels and the new Axel has been assigned a UID 3.  After
> > > > commit(), the new Axle is added to the Database (good) 4.
> > > The entity
> > > > is updated in the DataCache (I would assume this is good as
> > > > well)
> > > > 5.  "object" however now only contains one element in the
> > > Set<Axle> -
> > > > the one we added, the others are no longer there.
> > > > 6.  A subsequent refresh of the returns the same results as
> > > in #5! It
> > > > hits in the DataCache, so I assume that the Update of the Cache
> > > > contains only the "new" data from object.
> > > >
> > > > Turning off the DataCache (or excluding the Equipment
> > > Entity) solves
> > > > the problem.
> > > >
> > > > Here the complete config:
> > > >
> > > > openjpa.AutoClear: 0
> > > > openjpa.AutoDetach: [Ljava.lang.String;@60cf710e
> > > > openjpa.BrokerFactory: jdbc
> > > > openjpa.BrokerImpl: default
> > > > openjpa.CacheDistributionPolicy: default
> > > > openjpa.Callbacks: default
> > > > openjpa.ClassResolver: default
> > > > openjpa.Compatibility: default
> > > > openjpa.ConnectionDriverName: org.postgresql.Driver
> > > > openjpa.ConnectionFactoryMode: false
> > > > openjpa.ConnectionFactoryProperties: QueryTimeOut=5000,
> > > > PrettyPrint=true, PrettyPrintLineLength=80, PrintParameters=true
> > > > openjpa.ConnectionPassword: ******
> > > > openjpa.ConnectionProperties: MaxActive=100, MaxIdle=5, MinIdle=2,
> > > > MaxWait=60000
> > > > openjpa.ConnectionRetainMode: 0
> > > > openjpa.ConnectionURL: ******
> > > > openjpa.ConnectionUserName: *****
> > > > openjpa.DataCache: true(CacheSize=5000, EnableStatistics=true)
> > > > openjpa.DataCacheManager: default
> > > > openjpa.DataCacheTimeout: -1
> > > > openjpa.DetachState: fgs(DetachedStateField=true)
> > > > openjpa.DynamicDataStructs: false
> > > > openjpa.DynamicEnhancementAgent: true
> > > > openjpa.EntityManagerFactory: default
> > > > openjpa.FetchBatchSize: -1
> > > > openjpa.FetchGroups: [Ljava.lang.String;@53077fc9
> > > > openjpa.FlushBeforeQueries: 0
> > > > openjpa.Id: g11.persistence
> > > > openjpa.IgnoreChanges: false
> > > > openjpa.InitializeEagerly: false
> > > > openjpa.InstrumentationManager: default
> > > > openjpa.InverseManager: true
> > > > openjpa.LifecycleEventManager: validating
> > > > openjpa.LockManager: mixed
> > > > openjpa.Log: log4j
> > > > openjpa.ManagedRuntime: auto
> > > > openjpa.MaxFetchDepth: -1
> > > > openjpa.MetaDataFactory: *** truncated!!
> > > > openjpa.MetaDataRepository: default
> > > > openjpa.Multithreaded: false
> > > > openjpa.NontransactionalRead: true
> > > > openjpa.NontransactionalWrite: true
> > > > openjpa.Optimistic: true
> > > > openjpa.OrphanedKeyAction: log
> > > > openjpa.ProxyManager: default
> > > > openjpa.QueryCache: true(CacheSize=1000, SoftReferenceSize=100,
> > > > EvictPolicy='timestamp')
> > > > openjpa.QueryCompilationCache: all
> > > > openjpa.ReadLockLevel: 10
> > > > openjpa.RefreshFromDataCache: false
> > > > openjpa.RemoteCommitProvider: sjvm
> > > > openjpa.RestoreState: 1
> > > > openjpa.RetainState: true
> > > > openjpa.RetryClassRegistration: false
> > > > openjpa.RuntimeUnenhancedClasses: 1
> > > > openjpa.SavepointManager: in-mem
> > > > openjpa.Sequence: table
> > > > openjpa.TransactionMode: false
> > > > openjpa.WriteLockLevel: 20
> > > > openjpa.jdbc.DBDictionary:
> > > > postgres(supportsNullTableForGetImportedKeys=false)
> > > > openjpa.jdbc.DriverDataSource: auto
> > > > openjpa.jdbc.EagerFetchMode: 2
> > > > openjpa.jdbc.FetchDirection: 1000
> > > > openjpa.jdbc.FinderCache: true
> > > > openjpa.jdbc.IdentifierUtil: default
> > > > openjpa.jdbc.LRSSize: 2
> > > > openjpa.jdbc.MappingDefaults: jpa
> > > > openjpa.jdbc.QuerySQLCache: true(EnableStatistics=true)
> > > > openjpa.jdbc.ResultSetType: 1003
> > > > openjpa.jdbc.SQLFactory: default
> > > > openjpa.jdbc.SchemaFactory: native(ForeignKeys=true)
> > > > openjpa.jdbc.Schemas: [Ljava.lang.String;@60cf710e
> > > > openjpa.jdbc.SubclassFetchMode: 1
> > > > openjpa.jdbc.SynchronizeMappings: null
> > > > openjpa.jdbc.TransactionIsolation: -1
> > > > openjpa.jdbc.UpdateManager: default
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > 
> > > >
> > > > John
> > > >
> > > > ----
> > > >
> > > > Who is General Failure, and why is he reading my hard disk?
> > > >
> > > >
> > >
> >
>



-- 
*Rick Curtis*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message