batchee-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Manni-Bucau <rmannibu...@gmail.com>
Subject Re: JBatch shutdown / multipl Applications (was: Java7?)
Date Sun, 25 Mar 2018 16:37:35 GMT
Not 100% linked but did https://issues.apache.org/jira/browse/BATCHEE-132

Overall idea is to not get stucked to handle each service one by one and
have a single place to concentrate all cleanup tasks.

Side note on this JMX topic: each app should set application in batchee
properties.


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>

2018-03-23 15:14 GMT+01:00 Romain Manni-Bucau <rmannibucau@gmail.com>:

>
>
> 2018-03-23 14:33 GMT+01:00 Mark Struberg <struberg@yahoo.de.invalid>:
>
>>
>> Romain, I see what you mean.But I fear that simply doesn't work right
>> now.Case 2 is different. It means a central JobOperator for all WARs in the
>> server.But even if we would get different JobOperators then we still would
>> share the same database, right?We might introduce a column to differentiate
>> those. But that's something for the future.
>>
>>
>> Oki here is a case.
>> You have one app server. Might be TomEE, might be WAS9, Wildfly, etc
>> A WAR1 has batch1.WAR2 has batch2.
>> The app server starts. The first call to BatchRuntime.getJobOperator
>> resolves the JobOperator and registers the JMX Bean. Any other WAR which
>> starts and requests a JobOperator would then reuse the first one. Of course
>> in TomEE we would at least get different Services (at least I hope that).
>> When the first application stops we currently also close the active
>> BatchThreadService. Whilch is perfectly fine as it should only stop it's
>> own Jobs.But it also de-registers the JMX bean...
>> So the first app stop would kill JMX ;)
>>
>
> Sounds like an integration issue and not a Batchee issue. So jmx should be
> switched of or so no?
>
>
>> LieGrue,strub
>>     On Thursday, 22 March 2018, 14:22:55 CET, Romain Manni-Bucau <
>> rmannibucau@gmail.com> wrote:
>>
>>  1, 2 and 3 are the same case which is a container managing jbatch
>> runtime.
>> This is what we have in tomee with BatchEEServiceManager which should
>> @Observes BeforeApplicationDeployed to stop as you described the
>> application runtime (which is not shared between apps)
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github <
>> https://github.com/rmannibucau> |
>> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-
>> high-performance>
>>
>> 2018-03-22 13:04 GMT+01:00 Mark Struberg <struberg@yahoo.de.invalid>:
>>
>> >
>> > Indeed. Let me share the 3 different use cases we currently run JBatch
>> on
>> > 1.) WAS-8.5.5 (Java8, EE6) with BatchEE in a batch.war inside an EAR
>> > (shared backend jars in the EAR /lib folder).2.) TomEE-7.0.4 (Java8,
>> EE7)
>> > with BatchEE provided by TomEE3.) standalone batchee-cli with openejb
>> > lifecycle (again tomee-7.0.4 version).
>> >
>> > In case 1 and 3 we have the jbatch-api.jar in the WARs. So the static
>> > lookup of BatchRuntime.getJobOperator() will only find the
>> JobOperatorImpl
>> > provided inside the war.Shutting down the webapp should properly stop
>> all
>> > the Jobs which are currently running.
>> > In case 2 it is a bit more tricky. If the jbatch-api.jar is provided by
>> > the container we will really get a single JobOperatorImpl which is
>> shared
>> > across different WARs. And it must _not_ shut down when a single app
>> stops.
>> > The other implication is that the JobRepository is kind of shared
>> between
>> > different webapps then. We do not make any difference! Having war1 and
>> war2
>> > then both would see the jobs from the other app.
>> > But if a batch-job has a Batchlet which makes use of a CDI bean which
>> only
>> > exist in war2, then you are out of luck if you try to start it on
>> war1...
>> > LieGrue,strub
>> >
>> >    On Wednesday, 21 March 2018, 17:45:55 CET, Romain Manni-Bucau <
>> > rmannibucau@gmail.com> wrote:
>> >
>> >  2018-03-21 17:00 GMT+01:00 Mark Struberg <struberg@yahoo.de.invalid>:
>> >
>> > > > Previous impl was shutdown the executor, wait a bit and
>> > > > kill.
>> > > Yes, and exactly that caused some troubles in production ;)
>> > > Think about having a long running Batchlet. If you're lucky and do
>> some
>> > DB
>> > > reads or other IO which checks for Threads then you will get an
>> > > InterruptedException and fall on your noose the bloody way.If not it
>> > might
>> > > simply refuse to stop. And then the container might do a hard kill
>> after
>> > 5
>> > > minutes.
>> > > The idea is that our Container integration could notify all actively
>> > > running Batches to gracefully stop() before going forward to kill
>> them.
>> > > And while we have the container integration in TomEE and WebSphere it
>> > > might be good to have a portable way to do the same on standalone
>> > > containers, manually started batches, etc.
>> > >
>> >
>> > Yep, JBatch misses the notion of "environment" (or container). Having a
>> > long running JobOperator would be probably saner but I fear it is
>> already
>> > too spread out to modify it so extracting a JobRepository (application
>> > scoped) can be the way to go for spec N+1.
>> >
>> >
>> > > LieGrue,strub
>> > >
>> > >
>> > >    On Wednesday, 21 March 2018, 16:46:32 CET, Romain Manni-Bucau <
>> > > rmannibucau@gmail.com> wrote:
>> > >
>> > >  There are a few cases we can handle ourself probably:
>> > > 1. standalone -> shutdown hook (ensuring we remove it properly if we
>> > > shutdown manually before the end)2. webapp -> we already do our job3.
>> > > container -> container should handle it for you and not batchee4.
>> OSGi ->
>> > > we don't have an integration but the activator should do it
>> > > So I think we are not bad and dont need to make JobOperator which is a
>> > > prototype/request scope instancesomething which can impact the
>> > application
>> > > (scope).
>> > > @Scott: idea from mark is to propagate stop to batchlets etc to try a
>> > > graceful shutdown. Previous impl was shutdown the executor, wait a bit
>> > and
>> > > kill.
>> > > 2018-03-21 16:43 GMT+01:00 Scott Kurz <skurz3@gmail.com>:
>> > >
>> > > This is the first I can remember ever anyone mentioning standardizing
>> a
>> > > "stop" or "shutdown" of the batch runtime.
>> > >
>> > > I don't have much of an opinion at this point on whether it should be
>> > taken
>> > > forward in the standard.
>> > >
>> > > Since we're talking about it, I am curious what does BatchEE do here?
>> > >
>> > > Does it issue stop() on all running jobs?  Does it wait for them to
>> > finish?
>> > >  Would you expect some exception for any JobOperator call after
>> shutdown
>> > > (is it different depending on whether stop completes or not)?
>> > (Presumably
>> > > non-standard).
>> > >
>> > > Don't go through any great trouble answering..just throwing out some
>> > > thoughts that might need to be answered.
>> > >
>> > > Scott
>> > >
>> > >
>> > >
>> > > On Wed, Mar 21, 2018 at 10:14 AM, Mark Struberg <struberg@yahoo.de>
>> > wrote:
>> > >
>> > > > Well, it's a design decision. This is about BATCHEE-131.
>> > > >
>> > > > With calling BatchRuntime.getJobOperator() we lazily start the
>> > > > BatchRuntime.
>> > > >
>> > > > But currently there is no way to do a proper shutdown in a portable
>> > way!
>> > > > The current shutdown relies on calling our _internal_
>> ThreadPoolService
>> > > > SPI shutdown() method.
>> > > >
>> > > > Having the BatchRuntime implement AutoCloseable might be a good
>> > solution.
>> > > > That means someone can code - in a totally portable way - the
>> following
>> > > to
>> > > > stop the batch runtime:
>> > > >
>> > > > JobOperator jobOperator = BatchRuntime.getJobOperator();
>> > > >
>> > > > if (jobOperator instanceof AutoCloseable) {
>> > > >  ((AutoCloseable) jobOperator).close();
>> > > > }
>> > > >
>> > > > Will commit that improvement soon.
>> > > >
>> > > > @Scott, do you think we should put this feature up for inclusion
>> into
>> > the
>> > > > JBatch-1.1 specification?
>> > > >
>> > > > LieGrue,
>> > > > strub
>> > > >
>> > > >
>> > > > On Wednesday, 21 March 2018, 13:48:56 CET, Romain Manni-Bucau <
>> > > > rmannibucau@gmail.com> wrote:
>> > > >
>> > > >
>> > > > +1 for java 8 as well today
>> > > >
>> > > > side note: we dont need it on job operator right?
>> > > >
>> > > >
>> > > > Romain Manni-Bucau
>> > > > @rmannibucau <https://twitter.com/ rmannibucau> |  Blog
>> > > > <https://rmannibucau.metawerx. net/> | Old Blog
>> > > > <http://rmannibucau.wordpress. com> | Github <https://github.com/
>> > > > rmannibucau> |
>> > > > LinkedIn <https://www.linkedin.com/in/ rmannibucau> | Book
>> > > > <https://www.packtpub.com/ application-development/java-
>> > > > ee-8-high-performance>
>> > > >
>> > > > 2018-03-21 13:44 GMT+01:00 Mark Struberg <struberg@yahoo.de.invalid
>> >:
>> > > >
>> > > > > Hi Ioan!
>> > > > >
>> > > > > Also an option. But there are still servers with Java7 around.And
>> we
>> > > > still
>> > > > > need to support it for TomEE7.
>> > > > > We might add Java8 soon but then we'd also revisit the whole
API I
>> > > guess.
>> > > > > We could probably drive this for an upcoming JBatch-1.1
>> > specification.
>> > > > > Anyway, thanks for your feedback, always welcome!
>> > > > >
>> > > > > LieGrue,strub
>> > > > >    On Wednesday, 21 March 2018, 13:37:14 CET, Ioan Eugen Stan
<
>> > > > > stan.ieugen@gmail.com> wrote:
>> > > > >
>> > > > >  Hi,
>> > > > >
>> > > > > I'm a bit more progresist and move to Java 8 as baseline.
>> > > > >
>> > > > > https://plumbr.io/blog/java/ java-version-and-vendor-data-
>> > > > > analyzed-2017-edition
>> > > > >
>> > > > > http://www.baeldung.com/java- in-2017
>> > > > >
>> > > > >
>> > > > > On 21.03.2018 14:32, Mark Struberg wrote:
>> > > > > > Hi folks!
>> > > > > > I just figured that BatchEE is still baseline 1.6.
>> > > > > > Do we want to raise this to at least Java7? ^^
>> > > > > > Reason is that I want to use the AutoCloseable interface
for
>> proper
>> > > > > shutdown of the JobOperator.
>> > > > > > Any thoughts?
>> > > > > > Going for TLP and batchee-1.0?well new thread ;)
>> > > > > > LieGrue,strub
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message