deltaspike-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Schäfer, Johannes <jschae...@psi.de>
Subject RE: Performance of DeltaSpike Data
Date Thu, 15 Jun 2017 12:39:37 GMT
Hi,

Thanks for your investigation. I tried it today again with the last version from Github. For
my test setup I still have the same numbers. So in average DeltaSpike needs 3 times longer
that a pure EM implementation for finding entities by a query.
For now I stopped my evaluation of DeltaSpike Data. My personal feeling is quite mixed. I
really like to concept of DeltaSpike Data, but the performance loss is sometimes quite too
big.
I hope that there will be in future some improvements.

Regards
Johannes

-----Original Message-----
From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com] 
Sent: Friday, June 9, 2017 4:40 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data

FYI:

i created a similar example based on the unittests in the ds data module.
Here is the result of 100.000 selects, running on tomee and hsqldb:

DS:  2600ms
JPA: 1800ms


200-300ms are taken by our interceptor lookup, which can be improved a bit by caching the
information about interceptors in the proxy instance instead of doing a bean+cache lookup
each time.
another 200-300ms are taken by DS Data directly - not sure if we can improve it further.

Not sure about the other 200-400ms.

2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de>:

> No chances with @ApplicationScoped.
> To run my test just run "mvn clean install -PEmbeddedTestWeld".
> It is using a in memory H2 DB. Maybe pipe the output into a log to 
> find the test results in the output.
>
> Grüße
> Johannes
>
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 3:12 PM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> Could you also try to make your repository @ApplicationScoped?
>
> How can i run your tests? I don't like to install a AS or database ;)
>
> 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de>:
>
> > Same with a locally compile version.
> > Mysql:
> > ____________________________________________________________
> > ____________________________________________________________
> > _____________________________
> > |   | iter 10    | iter 20    | iter 40    | iter 80    | iter 160   |
> > iter 320   | iter 640   | iter 1280  | iter 2560  | iter 5120   | iter
> > 10240  |
> > |===========================================================
> > ============================================================
> > =============================|
> > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| 
> > | DS| 0.385907351|
> > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392| 
> > 25.372525787|
> > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| 
> > | EM| 0.195617863|
> > 0.386509868| 0.812829151| 1.4044238  | 3.007676477| 6.232350452 | 
> > 11.726264467|
> >
> > Grüße
> > Johannes
> >
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > Sent: Thursday, June 8, 2017 1:12 PM
> > To: users@deltaspike.apache.org
> > Subject: Re: Performance of DeltaSpike Data
> >
> > please build DS from source, i don't think that SNAPSHOT is up to date.
> >
> > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de>:
> >
> > > Hi,
> > >
> > > Thanks for your great support. Now I had the time to run the tests.
> > > Unfortunately no improvement. :-(
> > > I used Mysql and H2 and both still have a significant difference.
> > > mysql:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > |   | iter 10    | iter 20    | iter 40    | iter 80    | iter 160   |
> > > iter 320   | iter 640   | iter 1280  | iter 2560  | iter 5120   | iter
> > > 10240  |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| 
> > > | DS| 0.489673972|
> > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859| 
> > > 24.631240985|
> > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 
> > > | EM| 0.15662194
> > > | EM| |
> > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 | 
> > > 10.856276852|
> > >
> > > h2:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > __________________________
> > > |   | iter 10    | iter 20    | iter 40    | iter 80    | iter 160   |
> > > iter 320   | iter 640  | iter 1280  | iter 2560  | iter 5120  | iter
> > 10240 |
> > > |===========================================================
> > > ============================================================
> > > ==========================|
> > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| 
> > > | DS| 0.037355253|
> > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893| 
> > > 0.848326297|
> > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079  | 0.001897262| 
> > > | EM| 0.003144109|
> > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248| 
> > > 0.217664259|
> > >
> > > I used version 1.8.1-SNAPSHOT for testing.
> > > See https://github.com/johannesschaefer/javaee_jsf_
> > > cdi_jpa_data_ds_project_template
> > >
> > > Grüße
> > > Johannes
> > >
> > > -----Original Message-----
> > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > > Sent: Thursday, June 8, 2017 10:36 AM
> > > To: users@deltaspike.apache.org
> > > Subject: Re: Performance of DeltaSpike Data
> > >
> > > I did a major improvement and in my tests, both plain JPA and DS 
> > > Data are now very similar.
> > > Would be great if you could provide the new numbers.
> > >
> > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko 
> > > <andraschko.thomas@gmail.com
> > > >:
> > >
> > > > Hi,
> > > >
> > > > could you please try to run your test again against the github
> master?
> > > > I already did a small improvement and refactored a little bit on 
> > > > the weekend.
> > > >
> > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de>:
> > > >
> > > >> Hi,
> > > >>
> > > >> So after the a long weekend, I'm back with my results.
> > > >> For the write, findByPK and findAll tests I get now good numbers.
> > > >> See:
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/SaveTest.java
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadTest.java
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > > >>
> > > >> The difference between delta spike and plain EM are just a few 
> > > >> percent, in both directions ;-) .
> > > >>
> > > >> But I wrote a new test case were I try to find entities by an query.
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > > >> So I compare
> > > >>             TypedQuery< Material > query = eml.createQuery(
> > > >>                 "SELECT m FROM Material m WHERE grade = :grade 
> > > >> AND width = :width AND thickness = :thickness",
> > > >>                 Material.class );
> > > >>             query.setParameter( "grade", "AAA" );
> > > >>             query.setParameter( "width", 5 );
> > > >>             query.setParameter( "thickness", 5. ); List< 
> > > >> Material
> > > >> > mats = query.getResultList();
> > > >>
> > > >> to
> > > >> List< Material > mats = 
> > > >> matRepo.findByGradeAndWidthAndThickness(
> > > >> "AAA", 5, 5. );
> > > >>
> > > >> Here again the difference is quite high.
> > > >> |   | iter 10    | iter 20    | iter 40    | iter 80    | iter 160
>  |
> > > >> iter 320   | iter 640   | iter 1280  | iter 2560  | iter 5120   |
> iter
> > > >> 10240  |
> > > >> |===========================================================
> > > >> ============================================================
> > > >> =============================|
> > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| 
> > > >> | DS| 0.526700787|
> > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 
> > > >> 13.963582543| 26.785764953|
> > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| 
> > > >> | EM| 0.171045079|
> > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 
> > > >> | 12.361550486|
> > > >>
> > > >> So as you can see the DeltaSpike implementation needs at least 
> > > >> the double amount of time.
> > > >>
> > > >> Any hints for improving the performance?
> > > >>
> > > >> Regards,
> > > >> Johannes
> > > >>
> > > >> -----Original Message-----
> > > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> > > >> Sent: Thursday, June 1, 2017 2:27 PM
> > > >> To: users@deltaspike.apache.org
> > > >> Subject: RE: Performance of DeltaSpike Data
> > > >>
> > > >> Right. Copy and paste error.
> > > >> I added also a flush to the EM test.
> > > >> Now I have similar numbers.
> > > >> ____________________________________________________________
> > > >> ____________________________________________________________
> > > >> ______________________________
> > > >> |   | iter 10    | iter 20    | iter 40    | iter 80    | iter 160
>  |
> > > >> iter 320   | iter 640   | iter 1280  | iter 2560  | iter 5120   |
> iter
> > > >> 10240   |
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |=======|
> > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| 
> > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004| 
> > > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| 
> > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769| 
> > > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > > >>
> > > >> It's a little bit strange for me, why I have to compare 
> > > >> EntityPersistenceRepository.save with a em.persist + em.flush. 
> > > >> I would expect that an simple EntityPersistenceRepository.save 
> > > >> don't have a flush (there is a separate EntityPersistenceRepository.
> > > saveAndFlush).
> > > >>
> > > >> When I do a run with EntityPersistenceRepository.saveAndFlush I 
> > > >> get the following numbers.
> > > >> ____________________________________________________________
> > > >> ____________________________________________________________
> > > >> ______________________________
> > > >> |   | iter 10    | iter 20    | iter 40    | iter 80    | iter 160
>  |
> > > >> iter 320   | iter 640   | iter 1280  | iter 2560  | iter 5120   |
> iter
> > > >> 10240   |
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |=======|
> > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| 
> > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685| 
> > > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| 
> > > >> | EM| 0.031066334| 0.110747277| 0.4051742  | 1.925682412| 
> > > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > > >>
> > > >> So I have the feeling that there is still something wrong.
> > > >>
> > > >> Thanks to Gerhard for his additional hints.
> > > >> I committed all changes to the github repo.
> > > >>
> > > >> Regards,
> > > >> Johannes
> > > >>
> > > >> -----Original Message-----
> > > >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> > > >> Sent: Thursday, June 1, 2017 1:21 PM
> > > >> To: users@deltaspike.apache.org
> > > >> Subject: Re: Performance of DeltaSpike Data
> > > >>
> > > >> @johannes:
> > > >> as mentioned yesterday you have to move EntityTransaction#begin 
> > > >> and EntityTransaction#commit into the for-loop.
> > > >>
> > > >> regards,
> > > >> gerhard
> > > >>
> > > >>
> > > >>
> > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko 
> > > >> <andraschko.thomas@gmail.com
> > > >> >:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > ~1 year ago i did many optimizations in the data module and 
> > > >> > AFAIR DS Data was only a little bit slower.
> > > >> > After i compared my testcase with a benchmark from a user, 
> > > >> > the big different came from the transaction handling which 
> > > >> > was different in both testcases.
> > > >> >
> > > >> > Regards,
> > > >> > Thomas
> > > >> >
> > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek 
> > > >> > <gpetracek@apache.org
> >:
> > > >> >
> > > >> > > hi johannes,
> > > >> > >
> > > >> > > after refactoring your initial code to ds-test-control i

> > > >> > > saw
> e.g.
> > > >> > > ~6s vs 7,5s for 2560 iterations.
> > > >> > > i'll compare my local version with your new version 
> > > >> > > (mentioned in your mail).
> > > >> > >
> > > >> > > regards,
> > > >> > > gerhard
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes 
> > > >> > > <jschaefer@psi.de
> >:
> > > >> > >
> > > >> > > > Hi,
> > > >> > > >
> > > >> > > > My company is thinking about using DeltaSpike Data.
But 
> > > >> > > > before we integrate this into our development I was
asked 
> > > >> > > > to prepare some
> > > >> > > benchmarks,
> > > >> > > > comparing the usage of DeltaSpike Data with the usage
of 
> > > >> > > > a plain EntityManager.
> > > >> > > > I prepared some benchmarks and I was surprised that
there 
> > > >> > > > is a big difference in the write performance. I already

> > > >> > > > got some hints in the
> > > >> > > delta
> > > >> > > > spike irc channel, but the performance is still bad.
> > > >> > > > Based on a template from os890 I implemented my tests
and 
> > > >> > > > prepared a github project.
> > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > >> > cdi_jpa_data_ds_project_
> > > >> > > > template
> > > >> > > > Basically I'm talking about this test:
> > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > >> > cdi_jpa_data_ds_project_
> > > >> > > > template/blob/master/src/test/java/de/psi/metals/futurela
> > > >> > > > b/ repo/benchmark/SaveTest.java
> > > >> > > >
> > > >> > > > It just saves an entity into a DB in a loop. Depending
of 
> > > >> > > > the number of iterations the difference is quite big.
> > > >> > > >
> > > >> > > > SaveTest
> > > >> > > > _________________________________________________________
> > > >> > > > __
> > > >> > > > _
> > > >> > > > _________________________________________________________
> > > >> > > > __ _ _____________________________
> > > >> > > > |   | iter 10    | iter 20    | iter 40    | iter 80
   | iter
> > 160
> > > >>  |
> > > >> > > > iter 320   | iter 640   | iter 1280  | iter 2560  |
iter 5120
> >  |
> > > >> iter
> > > >> > > > 10240  |
> > > >> > > > |========================================================
> > > >> > > > |==
> > > >> > > > |=
> > > >> > > > =========================================================
> > > >> > > > == = =============================|
> > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|

> > > >> > > > | DS| 0.043319612|
> > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|

> > > >> > > > 20.920141274| 93.631768475|
> > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|

> > > >> > > > | EM| 0.028243393|
> > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > > >> > > > 0.745850523
> > > >> > > > |
> > > >> > > > 0.853543234 |
> > > >> > > >
> > > >> > > > Also the difference between a run with 5120 and 10240

> > > >> > > > iteration is not just the double amount of time, it
is 
> > > >> > > > more than 4 times
> > > more.
> > > >> > > >
> > > >> > > > Do you have some hints to me what I'm doing wrong there?
> > > >> > > >
> > > >> > > > Regards
> > > >> > > > Johannes
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Mime
View raw message