ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anton Vinogradov <avinogra...@gridgain.com>
Subject Re: Add emergency node closing handler to public Ignite API
Date Tue, 14 Nov 2017 18:05:49 GMT
Vova,

We should provide user ability to be notified in case some node decided to
stop itself.
Only user know how he want to be notified, so we should provide ability to
register custom callback(eg. send sms or call rest service)
This will cover cases when node stops gracefuly.

Please, see Semen's comment at
https://issues.apache.org/jira/browse/IGNITE-5811 for details.

P.s. Cases when node stops without ability to do something should be
covered by external watchdog.

Вт, 14 нояб. 2017 г. в 20:08, Vladimir Ozerov <vozerov@gridgain.com>:

> Can you explain what kind of logic could be placed there? And why do we
> need another configuration property and/or interface? We already have
> LifecycleBean, where Ignite instance could be injected, so user is already
> able to perform anything there.
>
> On Tue, Nov 14, 2017 at 7:46 PM, Anton Vinogradov <
> avinogradov@gridgain.com>
> wrote:
>
> > Vova,
> >
> > That's not about "kill -9" or OOM, that's about case when node detected
> > something and decided to stop itself (eg. persistence errors,
> > IgniteOutOfMemoryException, ExchangeWorker died)
> > Sure, we can't handle OOM or 100% CPU utilization by GC it that way, but
> we
> > can handle some logical problems.
> >
> > Andrey,
> >
> > I propose to refactor method to ignite.onClose(SomeClosure<SomeReason>)
> > In this case user will be able to register callback on all graceful
> stops,
> > and
> > detect it's reason.
> >
> >
> > On Tue, Nov 14, 2017 at 7:29 PM, Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> > > I am not sure this makes sense. First, in general case we do not have
> > > access to Java. E.g. in case of very long GC pause all Java threads are
> > > stuck and it is impossible to invoke anything. Second, some other
> > > conditions may be unrecoverable, such as OOME, where there is no
> > guarantee
> > > that any operation succeed. So this is not graceful shutdown. We should
> > > kill the node forcefully IMO.
> > >
> > > On Tue, Nov 14, 2017 at 7:23 PM, Andrey Kuznetsov <stkuzma@gmail.com>
> > > wrote:
> > >
> > > > Hi Igniters!
> > > >
> > > > When some node detects critical error, e.g. OOME, deadlock, etc, it
> > > should
> > > > invoke some user-defined callback and then attempt to close itself
> > > > gracefully. In order to make this possible we need to enhance Ignite
> > > > interface by adding something like Ignite.onEmergencyClose(
> > SomeClosure).
> > > >
> > > > First, I'd like to get your feedback on this potential change. Then
> we
> > > can
> > > > refine SomeClosure structure.
> > > >
> > > > --
> > > > Best regards,
> > > >   Andrey Kuznetsov.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message