cayenne-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Gentry <blackn...@gmail.com>
Subject Re: Cayenne object storage / memory usage
Date Thu, 20 Jul 2017 11:28:45 GMT
Maybe one of these days I'll get on Twitter...


On Thu, Jul 20, 2017 at 7:13 AM, Andrus Adamchik <andrus@objectstyle.org>
wrote:

> I did mention that it is coming: https://twitter.com/andrus_a/
> status/887931800705798144
>
> > On Jul 20, 2017, at 1:59 PM, Michael Gentry <blacknext@gmail.com> wrote:
> >
> > Sounds sweet.  Would be a great blog post on our new website design once
> > that happens, too.  Then tweet to the blog link...
> >
> > I'm jealous.  Still getting 3.1 polished off here...
> >
> >
> > On Thu, Jul 20, 2017 at 2:50 AM, Andrus Adamchik <andrus@objectstyle.org
> >
> > wrote:
> >
> >> Ok, some data on the new DataObject structure based on the testing with
> a
> >> real application.
> >>
> >> 1. UPGRADE:
> >>
> >> TL;DR: If you are not doing anything fancy, the upgrade is just
> >> regenerating Java classes with a new template. For special cases read
> on...
> >>
> >> I did an upgrade of a large old monolithic system, which is a bit too
> cosy
> >> with the old CayenneDataObject structure. It uses every single utility
> from
> >> cayenne-lifecycle, calls generic property API a lot, and otherwise takes
> >> advantage of the underlying Map structure. It was a good system to test
> >> this upgrade. Here are the instructions based on that experience beyond
> >> rerunning cgen:
> >>
> >> * Any vars declared as CayenneDataObject need to be replaced with just
> >> DataObject. The new object is still a DataObject, but inherits from the
> new
> >> BaseDataObject.
> >> * Superclass of any custom superclasses of the app persistent objects
> >> needs to be changed from CayenneDataObject to BaseDataObject.
> >> * Check all direct invocations of 'read|writeProperty[Directly]'. If
> all
> >> of them are using ORM-mapped property names, you are good.  Otherwise
> you
> >> will need to redefined these methods to fall back to a Map on unknown
> >> property. E.g. put [1] in the custom superclass. Going forward I think
> we
> >> may fold this code in a Cayenne superclass (HybridDataObject? :))
> >> * One particularly nasty extension in cayenne-lifecycle was the one
> >> handling "UUID relationships" (ObjectIdRelationshipHandler and friends
> ...
> >> hopefully not many people are using this). For each such relationship I
> had
> >> to create an ugly hack [2]. But it seems to work.
> >>
> >> 2. PERFROMANCE
> >>
> >> Now the exciting part. For performance testing I picked a monolithic
> >> read-only web service app with dozens (hundreds?) of endpoints.
> Essentially
> >> a huge query cache constantly which is refreshed non-stop via Cayenne
> >> queries. Lots of object churn and GC. An ideal app to test memory
> >> improvements, and the new structures did not disappoint. My benchmark
> >> compared the same app running under Cayenne 4.0.B1 (old) and 4.1 with
> >> field-based objects patch (new) on Java 7 and Jetty. The app was warmed
> up
> >> to account for class loading and cache initialization, and was then
> >> bombarded with HTTP requests for some time. The results:
> >>
> >> * Memory use: new is 49% less than old.
> >> * Time spent in GC (per jstat tool): new is 43% less than old.
> >> * Throughput: new is 27% higher (and climbing as the load rises).
> >>
> >> Looks impressive! Mind that these numbers are for the entire web app.
> >> Though query cache takes probably 90% of the app memory, so Cayenne
> >> optimization is having such a huge overall impact. The memory use drop
> >> helped in more than one way (can run on a smaller server; less GC means
> >> faster average response times and higher throughput). Just think how
> much
> >> money you can save on AWS costs! :)
> >>
> >> So here is my +1 on making field-based DataObject the default in 4.1.
> >>
> >> Andrus
> >>
> >> -------
> >> [1]
> >>
> >> private Map<String, Object> values;
> >>
> >> @Override
> >> public Object readPropertyDirectly(String propName) {
> >>        return values != null ? values.get(propName) : null;
> >> }
> >>
> >> @Override
> >> public void writePropertyDirectly(String propName, Object val) {
> >>
> >>        // no synchronization .. this is used for special cases and is
> >> hopefully single-threaded
> >>        if(values == null) {
> >>                values = new HashMap<>();
> >>        }
> >>
> >>        values.put(propName, val);
> >> }
> >>
> >> [2]
> >>
> >> private Factory _uuidFactory;
> >>
> >> @Override
> >> public void writePropertyDirectly(String propName, Object val) {
> >>    if(UUID_PROPERTY.equals(propName)) {
> >>        if(val instanceof Factory) {
> >>            _uuidFactory = (Factory) val;
> >>            uuid = null;
> >>            return;
> >>        }
> >>        else {
> >>            _uuidFactory = null;
> >>            uuid = (String) val;
> >>        }
> >>    }
> >>
> >>    super.writePropertyDirectly(propName, val);
> >> }
> >>
> >> @Override
> >> public Object readPropertyDirectly(String propName) {
> >>
> >>    if(UUID_PROPERTY.equals(propName)) {
> >>        if(_uuidFactory != null) {
> >>            return _uuidFactory;
> >>        }
> >>    }
> >>
> >>    return super.readPropertyDirectly(propName);
> >> }
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message