batchee-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Struberg <strub...@yahoo.de>
Subject Re: TomEE mem leak using batchee with JTA transactions
Date Thu, 05 Mar 2015 06:49:17 GMT
Yes, the main benefit of the entitymanager-per-request is if you e.g. use JSF and like to have
lazy loading in your render-response phase.
Or if you use JSP you touch entity methods in your tags or rendering without having the whole
page in a big transaction wrapper.

If you use DTOs then it does not add much benefit. But be aware that dealing with DTOs can
be _very_ tricky. E.g. you do not always get the id and version when you build your DTOs (but
only at the time the flush to the db happens in JPA). So your application might be plastered
with em.flush() which can heavily slow down your app. 
You also have to manually do the optimistic locking check and ACTIVELY maintain the version
in your DTOs (which sometimes is pure pain).

Otoh it nicely integrates with EJB, CDI and batches.

LIeGrue,
strub




> Am 04.03.2015 um 22:06 schrieb Karl Kildén <karl.kilden@gmail.com>:
> 
> OK. So I am not sure if I should use that module or not :)
> 
> I will remove requestscoped first thing tomorrow. The thing is I was thinking about doing
the thing where you inject the EntityManagerFactory instead and manually produce entitymanager
and give them requestscoped so the entitymanager would live for a jsf request. But then again
we are already used to the entities getting detached when leaving stateless so I will probably
never introduce it
> 
> On 4 March 2015 at 17:07, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> batchee-ee6 is the one, using an ejb so request scoped is implicitely started. remove
it and you should get context not active exception. jbatch just use a plain old trheadpoolexecutor
> 
> 
> Romain Manni-Bucau
> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> 
> 2015-03-04 16:47 GMT+01:00 Karl Kildén <karl.kilden@gmail.com>:
> Well I have it @RequestScoped and @PersistenceContext because of a mistake and it works
everywhere including stateless and Jbatch and I do no tricks. I will however remove it and
try again because it does not make sense.
> 
> I copied by dependencies from Caroline or something: 
> 
> 		<dependency>
> 			<groupId>org.apache.geronimo.specs</groupId>
> 			<artifactId>geronimo-jbatch_1.0_spec</artifactId>
> 			<version>${jbatch-api.version}</version>
> 			<scope>compile</scope>
> 			<!-- this JSR spec API is not yet provided in our EE6 containers -->
> 		</dependency>
> 
> 		<dependency>
> 			<groupId>org.apache.batchee</groupId>
> 			<artifactId>batchee-jbatch</artifactId>
> 			<version>${batchee.version}</version>
> 		</dependency>
> 		<dependency>
> 			<groupId>org.apache.batchee</groupId>
> 			<artifactId>batchee-extras</artifactId>
> 			<version>${batchee.version}</version>
> 		</dependency>
> 		<dependency>
> 			<groupId>org.apache.batchee</groupId>
> 			<artifactId>batchee-jsefa</artifactId>
> 			<version>${batchee.version}</version>
> 		</dependency>
> 		<dependency>
> 			<groupId>org.apache.batchee</groupId>
> 			<artifactId>batchee-cdi</artifactId>
> 			<version>${batchee.version}</version>
> 		</dependency>
> 		<dependency>
> 			<groupId>org.apache.batchee</groupId>
> 			<artifactId>batchee-ee6</artifactId>
> 			<version>${batchee.version}</version>
> 		</dependency>
> 
> 
> 
> Does that mean I have that extra module that uses stateless instead activated or not?
Would be good to know how the batch threads are started...
> 
> On 4 March 2015 at 13:23, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> @Mark: this has no link with EE 6 or 7, this is just a feature you want - which is fine.
JBatch doesn't deal with request scoped at all for instance. That said for batches we have
@JobScoped and @StepScoped which are still exeprimental in batchee-cdi but can be more adapted.
I know you are used to it but I just find it a non-sense to have named request scoped something
which is not bound to any http request but that's another topic ;)
> 
> 
> Romain Manni-Bucau
> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> 
> 2015-03-04 13:11 GMT+01:00 Mark Struberg <struberg@yahoo.de>:
> I did not read the full thread, but @Stateless and a @RequestScoped EntityManager doesn’t
make sense.
> @Stateless basically _only_ works well with @PersistenceContext. If you use DeltaSpike
JPA then I’d rather use @AppliationScoped + @Transactional (from deltaspike, not the half-broken
one from EE7).
> 
> 
> The EE support module btw is not just for WAS - it’s for all environments which support
EE but not yet EE7. The point is that with wrapping new thread creating in @Asynchronous ejb
call you get all the ThreadLocals set up for free. And it’s even needed on some EE7 container
as the concurrency-utils spec doesn’t define that the Context for @RequestScoped needs to
get started. Some containers do it, others don’t…
> 
> LieGrue,
> strub
> 
> 
> 
> > Am 02.03.2015 um 22:46 schrieb Romain Manni-Bucau <rmannibucau@gmail.com>:
> >
> > Depend your conf for both but a thread stack will say you in 2s
> >
> > Le 2 mars 2015 22:35, <karl.kilden@gmail.com> a écrit :
> > Hrmm. Probably not. But maybe, I would expect a clear error message though? Maybe
some other pool like stateless? Or will it get tired of waiting and throw?
> >
> > Skickat från min iPhone
> >
> > 2 mar 2015 kl. 21:58 skrev Romain Manni-Bucau <rmannibucau@gmail.com>:
> >
> >> Full db connection pool?
> >>
> >> Le 2 mars 2015 21:04, "Karl Kildén" <karl.kilden@gmail.com> a écrit
:
> >> Hi Romain, I removed all @Async usage and now it's the request thread that hangs
:D
> >>
> >> Actually when I dump the thread it seems to work forever being here and there
inside Eclipselink internals. Wonder if I triggered some kind of endless loop. It looks like
it because my heap is going way up and down and I am the only one using the app and whatever
task I started should be done aaaages ago.
> >>
> >> Big help getting my attention away from batch and async :-)
> >>
> >> I will keep analyzing. If it's not local to my app I will try to reproduce it
in a sample (but it's always quite hard to do that :/)
> >>
> >> thanks again
> >>
> >>
> >>
> >> On 2 March 2015 at 20:07, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> >> yes surely
> >>
> >> if you can put some effort to create a github project it can really help since
we'll identify the issue really faster (and where it comes from ;))
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> >>
> >> 2015-03-02 19:32 GMT+01:00 Karl Kildén <karl.kilden@gmail.com>:
> >> Romain you are right I am to tired now... Maybe I am quite stupid for putting
@RequestScoped on it since that is how I used to do it when I did tomcat.  It should not even
do anything when I think about it.
> >>
> >> This problem seems very related to how I use @Async. Maybe I should move my
topic with a new mail to tomee list?
> >>
> >> On 2 March 2015 at 19:27, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> >> well
> >>
> >> deltaspike data doesn't want @RequestScoped, it just used the contextual entity
manager - this comes from what JSF guys do AFAIK.
> >>
> >> Wonder if you could reproduce it with OpenJPA or if it is due to the fact eclipselink
is storing itself a state somewhere. Any idea?
> >>
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> >>
> >> 2015-03-02 19:13 GMT+01:00 Karl Kildén <karl.kilden@gmail.com>:
> >> Romain,
> >>
> >> Deltaspike Data wants a @RequestScoped entityManager. If I want to use Data
module from my batches, how to combine that?
> >>
> >> Also, this whole problem seems linked to @Async not batch (I thought batch was
implemented with @Async)
> >>
> >> On 2 March 2015 at 18:50, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> >> batchee default impl shouldnt be @Async excepted if you imported the module
Mark added for WAS - but your thread naming is closer to tomee ;).
> >>
> >> batches are by design asynchronous so no need of @Async to launch them.
> >>
> >> then all depends your @requestScoped. if it matches nothing the container handles
(http request or synchronous ejb call) then you should handle it yourself but sounds like
a workaround more than a fix which would be using a correct scope.
> >>
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> >>
> >> 2015-03-02 18:44 GMT+01:00 Karl Kildén <karl.kilden@gmail.com>:
> >> I was wrong - this problem is in many other places not just batches!
> >>
> >> regarding batch:
> >>
> >> Interesting, I have not done anything (what I know) to enable requestscoped...
> >>
> >> I thought Mark once told me that the impl in batchee for creating threads is
actually @Asynchronous. I also kind of recall not getting any extra threads in my batchee
jobs until I increased the @Async thread pool.
> >>
> >> I do use @Async myself also here and there... In fact I think in one or two
cases Asynchronous will start the batch. I use <class>org.apache.deltaspike.jpa.impl.transaction.EnvironmentAwareTransactionStrategy</class>
> >>
> >> Then I use this producer:
> >>
> >>      @PersistenceContext(unitName = APP_NAME)
> >>      private EntityManager entityManager;
> >>
> >>      @Produces
> >>      @RequestScoped
> >>      protected EntityManager createEntityManager() {
> >>              return this.entityManager;
> >>      }
> >>
> >>
> >>
> >> And a normal stateless that uses either the entityManager or a repository from
deltaspike data (actually almost always the repository). This is the only way I produce entityManagers.
> >>
> >>
> >> Anyways my problem seems to be also in JSF @ViewScoped beans and whatnot. Can
it be that I must dispose my entitymanagers myself somehow?
> >>
> >>
> >>
> >> On 2 March 2015 at 18:15, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> >> Hmm
> >>
> >> for a batch this code doesnt mean anything - request scope. Did you hack something
around detaspike to make it working?
> >>
> >> If this entity manager is used in an EJB this should be fine, if not then you
need to ensure transaction are handled as you expect - should be the case with batchee but
doesnt cost anything to validate it .
> >>
> >> Finally do you use @Asynchronous in your code otherwise you shouldn't see it
> >>
> >>
> >>
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> >>
> >> 2015-03-02 18:10 GMT+01:00 Karl Kildén <karl.kilden@gmail.com>:
> >> Hello,
> >>
> >> I have some @Stateless that I use from batches. After the job has finished I
can see after a heap dump that the async thread seems to keep a reference to the RepeatableWriteUnitOfWork.
When I google I understand that this is the EclipseLink entitymanager and since nobody seems
to have called clear on it my heap is getting pretty full...
> >>
> >> I have defined my Batches with normal read process write. They are @Named and
simply inject my @Stateless. They @Stateless uses EntityManager and it is produced like this:
> >>
> >>      @PersistenceContext(unitName = APP_NAME)
> >>      private EntityManager entityManager;
> >>
> >>      @Produces
> >>      @RequestScoped
> >>      protected EntityManager createEntityManager() {
> >>              return this.entityManager;
> >>      }
> >>
> >>
> >> Not sure if I am missing some kind of disposal here?  I don't think so because
only the jobs get the UnitOfWork stuck on the heap.
> >>
> >> Not sure I understand any of this very well. I can just clearly see that my
entire heap is now RepeatableWriteUnitOfWork tied to @ASynchronous threads.
> >>
> >> My memory dump could of course be sent to someone or shared desktop if someone
want's to help me understand this... Or maybe a pointer on where to debug?
> >>
> >> cheers
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> 
> 
> 
> 
> 


Mime
View raw message