Return-Path: X-Original-To: apmail-batchee-user-archive@minotaur.apache.org Delivered-To: apmail-batchee-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4A11717832 for ; Wed, 4 Mar 2015 15:48:20 +0000 (UTC) Received: (qmail 87405 invoked by uid 500); 4 Mar 2015 15:48:20 -0000 Delivered-To: apmail-batchee-user-archive@batchee.apache.org Received: (qmail 87375 invoked by uid 500); 4 Mar 2015 15:48:20 -0000 Mailing-List: contact user-help@batchee.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@batchee.incubator.apache.org Delivered-To: mailing list user@batchee.incubator.apache.org Received: (qmail 87365 invoked by uid 99); 4 Mar 2015 15:48:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Mar 2015 15:48:20 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of karl.kilden@gmail.com designates 74.125.82.181 as permitted sender) Received: from [74.125.82.181] (HELO mail-we0-f181.google.com) (74.125.82.181) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Mar 2015 15:47:54 +0000 Received: by wesq59 with SMTP id q59so47263415wes.1 for ; Wed, 04 Mar 2015 07:47:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ygXLcRlHAuMmZ3+asnPEvaHTzcSGeysomniy83krZH0=; b=dtiVGdodGpL6WQv9nNAAined8JTwY+D+Ts++GNAsJO2tnbi3jrXtfGnuQjj7JRMwU3 vw9rMvtVWVT10ArFSlwqJztGt0ko9XnC3ghXDGoOzpiLj/Zi6MZB+CTbaUM2Ds4+7G2X KVorBYy2Mtt/1jSropLHFd/lJ0RP/KSXBYfP48Ok5/UkL10cdk3A5npjwOxJah0Lj6Jx 4vW4+yTGsAbGZz9Gk2RXRXSEEUiIrWZekYKc4rmZiDtvi+fFKFkuNgLH4aLyfg1klSYv WKxPOS/LYf2AUwuR+uI3I0QbSspX/n0RBI4Dsg1OnWTlQeuBEmTR4L7Kw7oboaVuq7dE 4ETg== MIME-Version: 1.0 X-Received: by 10.194.20.67 with SMTP id l3mr9244231wje.94.1425484073275; Wed, 04 Mar 2015 07:47:53 -0800 (PST) Received: by 10.27.135.67 with HTTP; Wed, 4 Mar 2015 07:47:53 -0800 (PST) In-Reply-To: References: <9BDE3863-6A87-48F2-B5E9-AFDE78274FE8@gmail.com> <04F19F60-50ED-4181-8B23-CA93A56DB290@yahoo.de> Date: Wed, 4 Mar 2015 16:47:53 +0100 Message-ID: Subject: Re: TomEE mem leak using batchee with JTA transactions From: =?UTF-8?B?S2FybCBLaWxkw6lu?= To: "user@batchee.incubator.apache.org" Content-Type: multipart/alternative; boundary=047d7b5d649ad24f780510785f3f X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d649ad24f780510785f3f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Well I have it @RequestScoped and @PersistenceContext because of a mistake and it works everywhere including stateless and Jbatch and I do no tricks. I will however remove it and try again because it does not make sense. I copied by dependencies from Caroline or something: org.apache.geronimo.specs geronimo-jbatch_1.0_spec ${jbatch-api.version} compile org.apache.batchee batchee-jbatch ${batchee.version} org.apache.batchee batchee-extras ${batchee.version} org.apache.batchee batchee-jsefa ${batchee.version} org.apache.batchee batchee-cdi ${batchee.version} org.apache.batchee batchee-ee6 ${batchee.version} Does that mean I have that extra module that uses stateless instead activated or not? Would be good to know how the batch threads are started..= . On 4 March 2015 at 13:23, Romain Manni-Bucau wrote: > @Mark: this has no link with EE 6 or 7, this is just a feature you want - > which is fine. JBatch doesn't deal with request scoped at all for instanc= e. > That said for batches we have @JobScoped and @StepScoped which are still > exeprimental in batchee-cdi but can be more adapted. I know you are used = to > it but I just find it a non-sense to have named request scoped something > which is not bound to any http request but that's another topic ;) > > > Romain Manni-Bucau > @rmannibucau | Blog > | Github > | LinkedIn > | Tomitriber > > > 2015-03-04 13:11 GMT+01:00 Mark Struberg : > >> I did not read the full thread, but @Stateless and a @RequestScoped >> EntityManager doesn=E2=80=99t make sense. >> @Stateless basically _only_ works well with @PersistenceContext. If you >> use DeltaSpike JPA then I=E2=80=99d rather use @AppliationScoped + @Tran= sactional >> (from deltaspike, not the half-broken one from EE7). >> >> >> The EE support module btw is not just for WAS - it=E2=80=99s for all env= ironments >> which support EE but not yet EE7. The point is that with wrapping new >> thread creating in @Asynchronous ejb call you get all the ThreadLocals s= et >> up for free. And it=E2=80=99s even needed on some EE7 container as the >> concurrency-utils spec doesn=E2=80=99t define that the Context for @Requ= estScoped >> needs to get started. Some containers do it, others don=E2=80=99t=E2=80= =A6 >> >> LieGrue, >> strub >> >> >> >> > Am 02.03.2015 um 22:46 schrieb Romain Manni-Bucau < >> rmannibucau@gmail.com>: >> > >> > Depend your conf for both but a thread stack will say you in 2s >> > >> > Le 2 mars 2015 22:35, a =C3=A9crit : >> > Hrmm. Probably not. But maybe, I would expect a clear error message >> though? Maybe some other pool like stateless? Or will it get tired of >> waiting and throw? >> > >> > Skickat fr=C3=A5n min iPhone >> > >> > 2 mar 2015 kl. 21:58 skrev Romain Manni-Bucau : >> > >> >> Full db connection pool? >> >> >> >> Le 2 mars 2015 21:04, "Karl Kild=C3=A9n" a = =C3=A9crit : >> >> Hi Romain, I removed all @Async usage and now it's the request thread >> that hangs :D >> >> >> >> Actually when I dump the thread it seems to work forever being here >> and there inside Eclipselink internals. Wonder if I triggered some kind = of >> endless loop. It looks like it because my heap is going way up and down = and >> I am the only one using the app and whatever task I started should be do= ne >> aaaages ago. >> >> >> >> Big help getting my attention away from batch and async :-) >> >> >> >> I will keep analyzing. If it's not local to my app I will try to >> reproduce it in a sample (but it's always quite hard to do that :/) >> >> >> >> thanks again >> >> >> >> >> >> >> >> On 2 March 2015 at 20:07, Romain Manni-Bucau >> wrote: >> >> yes surely >> >> >> >> if you can put some effort to create a github project it can really >> help since we'll identify the issue really faster (and where it comes fr= om >> ;)) >> >> >> >> >> >> Romain Manni-Bucau >> >> @rmannibucau | Blog | Github | LinkedIn | Tomitriber >> >> >> >> 2015-03-02 19:32 GMT+01:00 Karl Kild=C3=A9n : >> >> Romain you are right I am to tired now... Maybe I am quite stupid for >> putting @RequestScoped on it since that is how I used to do it when I di= d >> tomcat. It should not even do anything when I think about it. >> >> >> >> This problem seems very related to how I use @Async. Maybe I should >> move my topic with a new mail to tomee list? >> >> >> >> On 2 March 2015 at 19:27, Romain Manni-Bucau >> wrote: >> >> well >> >> >> >> deltaspike data doesn't want @RequestScoped, it just used the >> contextual entity manager - this comes from what JSF guys do AFAIK. >> >> >> >> Wonder if you could reproduce it with OpenJPA or if it is due to the >> fact eclipselink is storing itself a state somewhere. Any idea? >> >> >> >> >> >> >> >> Romain Manni-Bucau >> >> @rmannibucau | Blog | Github | LinkedIn | Tomitriber >> >> >> >> 2015-03-02 19:13 GMT+01:00 Karl Kild=C3=A9n : >> >> Romain, >> >> >> >> Deltaspike Data wants a @RequestScoped entityManager. If I want to us= e >> Data module from my batches, how to combine that? >> >> >> >> Also, this whole problem seems linked to @Async not batch (I thought >> batch was implemented with @Async) >> >> >> >> On 2 March 2015 at 18:50, Romain Manni-Bucau >> wrote: >> >> batchee default impl shouldnt be @Async excepted if you imported the >> module Mark added for WAS - but your thread naming is closer to tomee ;)= . >> >> >> >> batches are by design asynchronous so no need of @Async to launch the= m. >> >> >> >> then all depends your @requestScoped. if it matches nothing the >> container handles (http request or synchronous ejb call) then you should >> handle it yourself but sounds like a workaround more than a fix which wo= uld >> be using a correct scope. >> >> >> >> >> >> >> >> Romain Manni-Bucau >> >> @rmannibucau | Blog | Github | LinkedIn | Tomitriber >> >> >> >> 2015-03-02 18:44 GMT+01:00 Karl Kild=C3=A9n : >> >> I was wrong - this problem is in many other places not just batches! >> >> >> >> regarding batch: >> >> >> >> Interesting, I have not done anything (what I know) to enable >> requestscoped... >> >> >> >> I thought Mark once told me that the impl in batchee for creating >> threads is actually @Asynchronous. I also kind of recall not getting any >> extra threads in my batchee jobs until I increased the @Async thread poo= l. >> >> >> >> I do use @Async myself also here and there... In fact I think in one >> or two cases Asynchronous will start the batch. I use >> org.apache.deltaspike.jpa.impl.transaction.EnvironmentAwareTransa= ctionStrategy >> >> >> >> Then I use this producer: >> >> >> >> @PersistenceContext(unitName =3D APP_NAME) >> >> private EntityManager entityManager; >> >> >> >> @Produces >> >> @RequestScoped >> >> protected EntityManager createEntityManager() { >> >> return this.entityManager; >> >> } >> >> >> >> >> >> >> >> And a normal stateless that uses either the entityManager or a >> repository from deltaspike data (actually almost always the repository). >> This is the only way I produce entityManagers. >> >> >> >> >> >> Anyways my problem seems to be also in JSF @ViewScoped beans and >> whatnot. Can it be that I must dispose my entitymanagers myself somehow? >> >> >> >> >> >> >> >> On 2 March 2015 at 18:15, Romain Manni-Bucau >> wrote: >> >> Hmm >> >> >> >> for a batch this code doesnt mean anything - request scope. Did you >> hack something around detaspike to make it working? >> >> >> >> If this entity manager is used in an EJB this should be fine, if not >> then you need to ensure transaction are handled as you expect - should b= e >> the case with batchee but doesnt cost anything to validate it . >> >> >> >> Finally do you use @Asynchronous in your code otherwise you shouldn't >> see it >> >> >> >> >> >> >> >> >> >> >> >> Romain Manni-Bucau >> >> @rmannibucau | Blog | Github | LinkedIn | Tomitriber >> >> >> >> 2015-03-02 18:10 GMT+01:00 Karl Kild=C3=A9n : >> >> Hello, >> >> >> >> I have some @Stateless that I use from batches. After the job has >> finished I can see after a heap dump that the async thread seems to keep= a >> reference to the RepeatableWriteUnitOfWork. When I google I understand t= hat >> this is the EclipseLink entitymanager and since nobody seems to have cal= led >> clear on it my heap is getting pretty full... >> >> >> >> I have defined my Batches with normal read process write. They are >> @Named and simply inject my @Stateless. They @Stateless uses EntityManag= er >> and it is produced like this: >> >> >> >> @PersistenceContext(unitName =3D APP_NAME) >> >> private EntityManager entityManager; >> >> >> >> @Produces >> >> @RequestScoped >> >> protected EntityManager createEntityManager() { >> >> return this.entityManager; >> >> } >> >> >> >> >> >> Not sure if I am missing some kind of disposal here? I don't think s= o >> because only the jobs get the UnitOfWork stuck on the heap. >> >> >> >> Not sure I understand any of this very well. I can just clearly see >> that my entire heap is now RepeatableWriteUnitOfWork tied to @ASynchrono= us >> threads. >> >> >> >> My memory dump could of course be sent to someone or shared desktop i= f >> someone want's to help me understand this... Or maybe a pointer on where= to >> debug? >> >> >> >> cheers >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > --047d7b5d649ad24f780510785f3f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Well I have it @RequestScoped and=C2=A0@PersistenceContext=C2=A0because of a mist= ake and it works everywhere including stateless and Jbatch and I do no tric= ks. I will however remove it and try again because it does not make sense.<= div>
I copied by dependencies from Caroline or something:=C2= =A0

<dep= endency>
= <groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-jba= tch_1.0_spec</artifactId>
<version>${jbatch-api.version}</version>
<scope>c= ompile</scope>
= <!-- this JSR spec API is not yet provided in our EE6 container= s -->
<= /dependency>

<dependency>
<groupId>org.apache.batchee</groupId><= /div>
<artifact= Id>batchee-jbatch</artifactId>
<version>${batchee.version}</version&g= t;
</depen= dency>
<= ;dependency>
<groupId>org.apache.batchee</groupId>
<artifactId>batchee-extras&= lt;/artifactId>
= <version>${batchee.version}</version>
</dependency>
<= span class=3D"" style=3D"white-space:pre"> <dependency>
=
<groupId>or= g.apache.batchee</groupId>
<artifactId>batchee-jsefa</artifactId>
<version>= ${batchee.version}</version>
</dependency>
<dependency>
<groupId>org.apache.batchee</= groupId>
= <artifactId>batchee-cdi</artifactId>
<version>${batchee.version}</= version>
&= lt;/dependency>
<= /span><dependency>
<groupId>org.apache.batchee</groupId>
<artifactId>batche= e-ee6</artifactId>
<version>${batchee.version}</version>
</dependency>
=



Does that mean I ha= ve that extra module that uses stateless instead activated or not? Would be= good to know how the batch threads are started...

On 4 March 2015 at 13:23, Roma= in Manni-Bucau <rmannibucau@gmail.com> wrote:
@Mark: this has no link with EE 6 = or 7, this is just a feature you want - which is fine. JBatch doesn't d= eal with request scoped at all for instance. That said for batches we have = @JobScoped and @StepScoped which are still exeprimental in batchee-cdi but = can be more adapted. I know you are used to it but I just find it a non-sen= se to have named request scoped something which is not bound to any http re= quest but that's another topic ;)


Romain Manni-Bucau
@rmannibucau | =C2=A0Blog=C2=A0| Github=C2=A0| LinkedIn=C2= =A0| Tomitriber<= /div>

2015-03-04 13:= 11 GMT+01:00 Mark Struberg <struberg@yahoo.de>:
I did not read the full thread, but @Stateless and a = @RequestScoped EntityManager doesn=E2=80=99t make sense.
@Stateless basically _only_ works well with @PersistenceContext. If you use= DeltaSpike JPA then I=E2=80=99d rather use @AppliationScoped + @Transactio= nal (from deltaspike, not the half-broken one from EE7).


The EE support module btw is not just for WAS - it=E2=80=99s for all enviro= nments which support EE but not yet EE7. The point is that with wrapping ne= w thread creating in @Asynchronous ejb call you get all the ThreadLocals se= t up for free. And it=E2=80=99s even needed on some EE7 container as the co= ncurrency-utils spec doesn=E2=80=99t define that the Context for @RequestSc= oped needs to get started. Some containers do it, others don=E2=80=99t=E2= =80=A6

LieGrue,
strub



> Am 02.03.2015 um 22:46 schrieb Romain Manni-Bucau <rmannibucau@gmail.com>:
>
> Depend your conf for both but a thread stack will say you in 2s
>
> Le 2 mars 2015 22:35, <karl.kilden@gmail.com> a =C3=A9crit :
> Hrmm. Probably not. But maybe, I would expect a clear error message th= ough? Maybe some other pool like stateless? Or will it get tired of waiting= and throw?
>
> Skickat fr=C3=A5n min iPhone
>
> 2 mar 2015 kl. 21:58 skrev Romain Manni-Bucau <rmannibucau@gmail.com>:
>
>> Full db connection pool?
>>
>> Le 2 mars 2015 21:04, "Karl Kild=C3=A9n" <karl.kilden@gmail.com&g= t; a =C3=A9crit :
>> Hi Romain, I removed all @Async usage and now it's the request= thread that hangs :D
>>
>> Actually when I dump the thread it seems to work forever being her= e and there inside Eclipselink internals. Wonder if I triggered some kind o= f endless loop. It looks like it because my heap is going way up and down a= nd I am the only one using the app and whatever task I started should be do= ne aaaages ago.
>>
>> Big help getting my attention away from batch and async :-)
>>
>> I will keep analyzing. If it's not local to my app I will try = to reproduce it in a sample (but it's always quite hard to do that :/)<= br> >>
>> thanks again
>>
>>
>>
>> On 2 March 2015 at 20:07, Romain Manni-Bucau <rmannibucau@gmail.com> wro= te:
>> yes surely
>>
>> if you can put some effort to create a github project it can reall= y help since we'll identify the issue really faster (and where it comes= from ;))
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau |=C2=A0 Blog | Github | LinkedIn | Tomitriber
>>
>> 2015-03-02 19:32 GMT+01:00 Karl Kild=C3=A9n <karl.kilden@gmail.com>:
>> Romain you are right I am to tired now... Maybe I am quite stupid = for putting @RequestScoped on it since that is how I used to do it when I d= id tomcat.=C2=A0 It should not even do anything when I think about it.
>>
>> This problem seems very related to how I use @Async. Maybe I shoul= d move my topic with a new mail to tomee list?
>>
>> On 2 March 2015 at 19:27, Romain Manni-Bucau <rmannibucau@gmail.com> wro= te:
>> well
>>
>> deltaspike data doesn't want @RequestScoped, it just used the = contextual entity manager - this comes from what JSF guys do AFAIK.
>>
>> Wonder if you could reproduce it with OpenJPA or if it is due to t= he fact eclipselink is storing itself a state somewhere. Any idea?
>>
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau |=C2=A0 Blog | Github | LinkedIn | Tomitriber
>>
>> 2015-03-02 19:13 GMT+01:00 Karl Kild=C3=A9n <karl.kilden@gmail.com>:
>> Romain,
>>
>> Deltaspike Data wants a @RequestScoped entityManager. If I want to= use Data module from my batches, how to combine that?
>>
>> Also, this whole problem seems linked to @Async not batch (I thoug= ht batch was implemented with @Async)
>>
>> On 2 March 2015 at 18:50, Romain Manni-Bucau <rmannibucau@gmail.com> wro= te:
>> batchee default impl shouldnt be @Async excepted if you imported t= he module Mark added for WAS - but your thread naming is closer to tomee ;)= .
>>
>> batches are by design asynchronous so no need of @Async to launch = them.
>>
>> then all depends your @requestScoped. if it matches nothing the co= ntainer handles (http request or synchronous ejb call) then you should hand= le it yourself but sounds like a workaround more than a fix which would be = using a correct scope.
>>
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau |=C2=A0 Blog | Github | LinkedIn | Tomitriber
>>
>> 2015-03-02 18:44 GMT+01:00 Karl Kild=C3=A9n <karl.kilden@gmail.com>:
>> I was wrong - this problem is in many other places not just batche= s!
>>
>> regarding batch:
>>
>> Interesting, I have not done anything (what I know) to enable requ= estscoped...
>>
>> I thought Mark once told me that the impl in batchee for creating = threads is actually @Asynchronous. I also kind of recall not getting any ex= tra threads in my batchee jobs until I increased the @Async thread pool. >>
>> I do use @Async myself also here and there... In fact I think in o= ne or two cases Asynchronous will start the batch. I use <class>org.a= pache.deltaspike.jpa.impl.transaction.EnvironmentAwareTransactionStrategy&l= t;/class>
>>
>> Then I use this producer:
>>
>>=C2=A0 =C2=A0 =C2=A0 @PersistenceContext(unitName =3D APP_NAME)
>>=C2=A0 =C2=A0 =C2=A0 private EntityManager entityManager;
>>
>>=C2=A0 =C2=A0 =C2=A0 @Produces
>>=C2=A0 =C2=A0 =C2=A0 @RequestScoped
>>=C2=A0 =C2=A0 =C2=A0 protected EntityManager createEntityManager() = {
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return this.entity= Manager;
>>=C2=A0 =C2=A0 =C2=A0 }
>>
>>
>>
>> And a normal stateless that uses either the entityManager or a rep= ository from deltaspike data (actually almost always the repository). This = is the only way I produce entityManagers.
>>
>>
>> Anyways my problem seems to be also in JSF @ViewScoped beans and w= hatnot. Can it be that I must dispose my entitymanagers myself somehow?
>>
>>
>>
>> On 2 March 2015 at 18:15, Romain Manni-Bucau <rmannibucau@gmail.com> wro= te:
>> Hmm
>>
>> for a batch this code doesnt mean anything - request scope. Did yo= u hack something around detaspike to make it working?
>>
>> If this entity manager is used in an EJB this should be fine, if n= ot then you need to ensure transaction are handled as you expect - should b= e the case with batchee but doesnt cost anything to validate it .
>>
>> Finally do you use @Asynchronous in your code otherwise you should= n't see it
>>
>>
>>
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau |=C2=A0 Blog | Github | LinkedIn | Tomitriber
>>
>> 2015-03-02 18:10 GMT+01:00 Karl Kild=C3=A9n <karl.kilden@gmail.com>:
>> Hello,
>>
>> I have some @Stateless that I use from batches. After the job has = finished I can see after a heap dump that the async thread seems to keep a = reference to the RepeatableWriteUnitOfWork. When I google I understand that= this is the EclipseLink entitymanager and since nobody seems to have calle= d clear on it my heap is getting pretty full...
>>
>> I have defined my Batches with normal read process write. They are= @Named and simply inject my @Stateless. They @Stateless uses EntityManager= and it is produced like this:
>>
>>=C2=A0 =C2=A0 =C2=A0 @PersistenceContext(unitName =3D APP_NAME)
>>=C2=A0 =C2=A0 =C2=A0 private EntityManager entityManager;
>>
>>=C2=A0 =C2=A0 =C2=A0 @Produces
>>=C2=A0 =C2=A0 =C2=A0 @RequestScoped
>>=C2=A0 =C2=A0 =C2=A0 protected EntityManager createEntityManager() = {
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return this.entity= Manager;
>>=C2=A0 =C2=A0 =C2=A0 }
>>
>>
>> Not sure if I am missing some kind of disposal here?=C2=A0 I don&#= 39;t think so because only the jobs get the UnitOfWork stuck on the heap. >>
>> Not sure I understand any of this very well. I can just clearly se= e that my entire heap is now RepeatableWriteUnitOfWork tied to @ASynchronou= s threads.
>>
>> My memory dump could of course be sent to someone or shared deskto= p if someone want's to help me understand this... Or maybe a pointer on= where to debug?
>>
>> cheers
>>
>>
>>
>>
>>
>>
>>
>>



--047d7b5d649ad24f780510785f3f--