ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: IgniteEvents for MVCC caches
Date Mon, 24 Sep 2018 11:53:34 GMT
Igniters,

Igniters, I see three distinct problems with cache events, if we are to put
SQL aside.

1) Strange "ENTRY_CREATED" and "ENTRY_DESTROYED" events
They simply look outdated. I propose to deprecate them in the next AI
release with optional flag to re-enalbe them if someone really used them.

2) Inconsistency for PUT/REMOVE/READ events
Currently for different transaction types these events are fired at
different times. For TRANSACTIONAL cache we fire them when entry is really
read or updated, what depends on transaction mode. E.g. for
PESSIMISTIC/READ_COMMITTED mode two subsequent get() will raise two events.
For PESSIMISTIC/REPEATABLE_READ it will be one event. The same goes for
puts. For MVCC caches it will work differently - every update will trigger
separate PUT event if executed through SQL request. But not every update
through PUT API will trigger an event. I think we need to come to clear
semantics here, which can be explained for user. For me it could sound like
this:
*"If entry is read inside a transaction, at least one read event is fired.
If entry is updated within transaction, at least one update event is
fired."*

What do you think?

3) Lock/unlock events inconsistency
>From what I know "LOCKED/UNLOCKED" events has no relation to
IgniteCache.lock() method family. These events are fired in any cache when
entry is locked for update. This is valid for all cache modes. But due to
arhcitecture differences, we cannot fire UNLOCK events for MVCC caches.
Because there is no explicit UNLOCK phase anymore. That is, we know when
current transaction locked it. But the only way to understand that entry is
unlocked is to check it in subsequent transaction. The only way to fix is
to store all affected keys in-memory, what will lead to OOME for large
transactions. My proposal here is not to fire "UNLOCK" event for MVCC
caches.

So, to summarize, my proposal is:
1) Deprecate ENTRY_CREATED/ENTRY_DESTROYED events
2) Formulate in JavaDocs semantics of PUT/REMOVE/READ events. MVCC doesn't
bring any new problems here.
3) Do not fire UNLOCK events for MVCC (explain in JavaDocs)

What do you think?

Vladimir.

On Mon, Sep 24, 2018 at 1:28 PM Павлухин Иван <vololo100@gmail.com> wrote:

> Hi guys,
>
> Andrey, thanks for pointing out explicit lock/unlock operations. Actually,
> I think that lock/unlock events are perfectly valid for TRANSACTIONAL
> caches, so there is a little chance that they could be deprecated. On the
> other hand mvcc lock/unlock is not related to non-mvcc lock/unlock because
> we do not support transactions involving caches with different modes. I
> could imagine that lock/unlock events can be used for debugging purposes.
> Currently events documentation is quite humble, e.g. for
> EVT_CACHE_OBJECT_LOCKED event we have javadoc "Built-in event type: object
> locked.". What object? What does "locked" mean? And "locking" semantics is
> not slightly different for mvcc and non-mvcc caches (already mentioned
> unlock on commit). So, another idea which came to my mind is introducing
> additional event type, say MVCC_LOCKED.
>
> Denis, there are 2 groups of cache events: EVT_CACHE_ENTRY_<...> and
> EVT_CACHE_OBJECT_<...>. EVT_CACHE_ENTRY_CREATED and
> EVT_CACHE_ENTRY_DESTROYED are related to on-heap part of cache behavior.
> Correct me if I am wrong, but now all caches use off-heap storage. And
> EVT_CACHE_ENTRY_CREATED, EVT_CACHE_ENTRY_DESTROYED are triggered when
> GridCacheMapEntry object is created as part of an access chain to values
> stored in the cache. E.g. cache.put invocation causes following sequence of
> events: CACHE_ENTRY_CREATED, CACHE_OBJECT_PUT, CACHE_ENTRY_DESTROYED.
> Actually, there are no special problems with supporting CACHE_ENTRY_<...>
> events for MVCC. And yes, I do not see much user value in them.
>
> Also, it seems that I chose a little bit confusing name for the thread. I
> would like to keep current discussion mainly about key-value API events for
> MVCC caches. SQL and events is a little bit different topic with number of
> additional problems.
>
> 2018-09-22 17:35 GMT+03:00 Denis Magda <dmagda@apache.org>:
>
> > Vladimir,
> >
> > That's a good question. I confused _ENTRY_ events with those mentioned by
> > you.
> >
> > Anyway, are you saying that the _ENTRY_ events are workable but have no
> > value or they are not workable any longer with the MVCC?
> >
> > --
> > Denis
> >
> >
> >
> > On Sat, Sep 22, 2018 at 4:10 AM Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> > > Denis,
> > >
> > > What is the point of these events, provided that we already has
> CACHE_PUT
> > > and CACHE_REMOVED?
> > >
> > > On Sat, Sep 22, 2018 at 1:01 AM Denis Magda <dmagda@apache.org> wrote:
> > >
> > > > Hello Ivan,
> > > >
> > > > First of all, it looks like that EVT_CACHE_ENTRY_CREATED,
> > > > > EVT_CACHE_ENTRY_DESTROYED are legacy came from Ignite 1.x time and
> > > have a
> > > > > little value nowadays. Does anyone have something against
> deprecating
> > > > them?
> > > >
> > > >
> > > > These events are used in a pure caching or in-memory data grid use
> > cases
> > > > where key-value is a primary access pattern. I personally know
> several
> > > > companies who rely on those events. So, we can't discontinue them,
> not
> > > > everyone uses SQL. What's a hurdle of making them working with MVCC?
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Fri, Sep 21, 2018 at 6:44 AM Павлухин Иван <vololo100@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > As you might know MVCC was introduced in Apache Ignite. We started
> > > > > IgniteEvents support implementation for MVCC caches and faced some
> > > > > obstacles. I would like to start a discussion about next steps
> which
> > > > should
> > > > > be done to deal with current problems. The main theme is about
> > > > > EventType.EVTS_CACHE and key-value API.
> > > > >
> > > > > First of all, it looks like that EVT_CACHE_ENTRY_CREATED,
> > > > > EVT_CACHE_ENTRY_DESTROYED are legacy came from Ignite 1.x time and
> > > have a
> > > > > little value nowadays. Does anyone have something against
> deprecating
> > > > them?
> > > > >
> > > > > Second, there could be problems with EVT_CACHE_OBJECT_UNLOCKED
> event,
> > > > > because for MVCC all locks are released on transaction end. And it
> > does
> > > > not
> > > > > sound good idea to track all locked entries during transaction
> > > execution,
> > > > > in MVCC we are not keeping entries modified participating in
> > > transaction
> > > > in
> > > > > private working set in memory. One possible solution is deprecation
> > of
> > > > > lock, unlock events. Another one is introducing special lock event
> > for
> > > > > MVCC, but it will be confusing. Also, I see an option of firing
> only
> > > > > EVT_CACHE_OBJECT_LOCKED.
> > > > >
> > > > > Last, is different number of events fired for similar scenarios and
> > > > > different cache modes. Let's consider "put, remove, put" for the
> same
> > > key
> > > > > and different modes. For ATOMIC 2 put and 1 remove event will be
> > fired.
> > > > For
> > > > > TRANSACTIONAL 1 put will be fired in case of commit and nothing in
> > case
> > > > of
> > > > > rollback. For MVCC in current vision 2 put will be fired regardless
> > > > whether
> > > > > transaction was committed and rolled back. Currently I do not see
> > > options
> > > > > how to overcome it.
> > > > >
> > > > > Also, I hardly imagine current use cases for cache events. I think
> > that
> > > > > understanding them is the best way for developing working solution
> > for
> > > > > MVCC.
> > > > >
> > > > > I need your opinions.
> > > > >
> > > > > 2018-09-21 12:54 GMT+03:00 Павлухин Иван <vololo100@gmail.com>:
> > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > As you might know MVCC was introduced in Apache Ignite. We
> started
> > > > > > IgniteEvents implementation for MVCC caches and faced some
> > > obstacles. I
> > > > > > would like to start a discussion about next steps which should
be
> > > done
> > > > to
> > > > > > deal with current problems.
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message