ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: Future of Ignite transactions
Date Tue, 26 Sep 2017 10:03:57 GMT
Dmitriy,

Agree with Vladimir here. OPTIMISTIC + (READ_COMMITTED | REPEATABLE_READ)
modes are completely unusable in real-life use-cases because they do not
allow any read-write conflict detection and thus the explicit transaction
statement can be omitted at all. The remaining combinations of tx
concurrency and isolation can be reduced to a new enum that will match SQL
isolation levels with crystal clear isolation guarantees.

I see no problem with deprecating the old API and completely removing it in
3.0.

2017-09-26 11:52 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:

> On Tue, Sep 26, 2017 at 12:08 AM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Dima,
> >
> > My proposal would require two changes:
> > 1) Public - for sure we will continue support old API
> >
>
> Of course. But my assumption is that you would deprecate the old API,
> right? If yes, then you will never be able to remove the deprecated API.
>
> 2) Internal TX engine - these changes are inevitable for transactional SQL
> > support.
> >
>
> Internally we can do any changes we like, as long as they don't change the
> expected user behavior.
>
>
> >
> > So we have no freedom to choose whether to spend time on internals or
> not.
> > We will have to do that anyway.
> >
> > On Tue, Sep 26, 2017 at 9:02 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > Vladimir,
> > >
> > > I do not think we have a luxury of changing Ignite transaction APIs. It
> > is
> > > almost the same as changing Ignite put and get APIs. Too much code has
> > > already been written to these APIs. The upgrade path will be so onerous
> > > that no one will ever take it.
> > >
> > > As far as the current transaction flags, we will have to continue to
> > > support them. Ideally we should extend them to Ignite SQL as well. If
> > there
> > > is a new transaction flag that you wish to add to Ignite SQL, please
> > > suggest it here.
> > >
> > > D.
> > >
> > > On Mon, Sep 25, 2017 at 10:48 PM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Dima,
> > > >
> > > > Becuase if you take any pair of our concurrency/isolation modes, you
> > will
> > > > not find anything similar in SQL. The main problem is that there is
> no
> > > > "OPTIMISTIC" or "PESSIMISTIC" in SQL world. All reads are
> "optimistic",
> > > but
> > > > can be converted to "perssimistic" optionally, and all writes are
> > > > "pessimistic". Another problem is that some of our pairs cannot be
> used
> > > in
> > > > practice (OPTIMISTIC/READ_COMMITTED, OPTIMISTIC/REPEATABLE_READ), and
> > > some
> > > > duplicates each other (PESSIMISTIC/REPEATABLE_READ -
> > > > PESSIMISTIC/SERIALIZABLE). This make our API very hard to explain and
> > > > reason about. If you cannot explain API in 5 mins, it is broken :-)
> > > >
> > > > If we were many years in the past allowed to choose between Ignite
> and
> > > SQL
> > > > approaches I would prefer SQL because it is much simpler.
> > > >
> > > > On Tue, Sep 26, 2017 at 1:29 AM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > This looks like a complete overhaul of our transactional behavior
> > and I
> > > > do
> > > > > not think we are at a liberty to make such drastic changes. Can you
> > > > please
> > > > > explain why the current behavior would not map to the SQL
> > transactions?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov <
> > > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > Sorry for late reply. I had a chat with several Ignite veterans
> > > today.
> > > > We
> > > > > > tried to design transactional SQL for Ignite. One of our
> questions
> > > was
> > > > > how
> > > > > > to align SQL transactions with current Ignite transactions.
And
> we
> > > > > failed.
> > > > > > And then we came to conclusion that current transaction API
is
> > > > unusable.
> > > > > We
> > > > > > have 6 pairs of modes from API standpoint and 4 real modes.
This
> is
> > > > very
> > > > > > counterintuitive and cannot be mapped to any transactional
> > framework
> > > > our
> > > > > > users are familiar with.
> > > > > >
> > > > > > So we thought how new tx API might looks like, and here is the
> > draft.
> > > > > >
> > > > > > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes
> > with
> > > > > > current enum) with three standard modes - READ_COMMITTED,
> > > > > REPEATABLE_READ,
> > > > > > SERIALIZABLE.
> > > > > > 2) Define new enum *TransactionHint* - READ_ONLY,
> > OPTIMISTIC_LOCKING
> > > > > > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously.
> > > > > > 4) Reads never acuire locks
> > > > > > 5) Writes always acquire locks
> > > > > > 6) *IgniteCache.withReadForUpdate()* will return special facade
> > > which
> > > > > will
> > > > > > obtain locks on reads. This is analogue of SELECT FOR UPDATE
in
> > SQL.
> > > > > > 7) *TransactionHint.READ_ONLY* - forces transaction to throw
an
> > > > exception
> > > > > > on any update
> > > > > > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction
into
> > our
> > > > > > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable
> only
> > > to
> > > > > > SERIALIZABLE isolation level.
> > > > > > 9) Define new API methods:
> > > > > > - IgniteTransactions.txStart(TransactionIsolationLevel
> isolation)
> > > > > > - IgniteTransactions.txStart(TransactionIsolationLevel
> isolation,
> > > > > > TransactionHint... hints)
> > > > > > 10) Deprecate old TX start methods
> > > > > >
> > > > > > As a result we will have simple, clean and extensible API. Which
> > can
> > > be
> > > > > > explained to users in 5 minutes, instead of current half an
hour.
> > And
> > > > > which
> > > > > > is perfectly aligned with upcoming transactional SQL.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 7, 2017 at 6:48 AM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Vova,
> > > > > > >
> > > > > > > Thanks for doing the research. The changes you are suggesting
> > are a
> > > > bit
> > > > > > too
> > > > > > > bold, so let's discuss them in some more detail...
> > > > > > >
> > > > > > > On Wed, Sep 6, 2017 at 4:51 AM, Vladimir Ozerov <
> > > > vozerov@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > We are moving towards DBMS system. None of them has
a notion
> of
> > > > > > > > OPTIMISTIC/PESSIMISTIC transactions. Instead, they
work as
> > > follows:
> > > > > > > > 1) Reads (SELECT) do not acquire exclusive row locks
> > > > > > > > 2) Exclusive lock on read could be forced explicitly
(SELECT
> > ...
> > > > FOR
> > > > > > > > UPDATE)
> > > > > > > > 3) Writes do acuire explicit row locks
> > > > > > > > 4) Locks are always acquired immediately once statement
is
> > > executed
> > > > > > > > 5) The strictest concurrency level - typically SERIALIZABLE
-
> > > rely
> > > > on
> > > > > > > > so-called *range locks* (or *predicate locks*) to
track
> > > > dependencies
> > > > > > > > between transactions. Some vendors throw an exception
in case
> > of
> > > > > > > conflict -
> > > > > > > > these are ones where snapshot-based MVCC is used -
> PostgreSQL,
> > > > > Oracle.
> > > > > > > > Others do aggressive locking - ones where two-phase
locking
> > > > algorithm
> > > > > > is
> > > > > > > > used - SQL Server, MySQL.
> > > > > > > >
> > > > > > > > As you see, there is no concept of PESSIMISTIC/OPTIMISTIC
> > modes.
> > > > > > Instead,
> > > > > > > > all updates are "PESSIMISTIC", reads are "OPTIMISTIC"
but
> could
> > > > > become
> > > > > > > > "PESSIMISTIC" if requested explicitly, and for snapshot-based
> > > > vendors
> > > > > > (we
> > > > > > > > are going in this direction) read-write conflicts
are
> resolved
> > in
> > > > > > manner
> > > > > > > > somewhat similar to our OPTIMISTIC/SERIALIZABLE.
> > > > > > > >
> > > > > > > > That said, I would propose to think on how transactions
could
> > > look
> > > > > like
> > > > > > > in
> > > > > > > > future Ignite versions (say, 3.0). My rough vision:
> > > > > > > >
> > > > > > > > 1) No OPTIMISTIC mode at all - too counterintuitive
and
> > complex.
> > > > It's
> > > > > > > only
> > > > > > > > advantage is deadlock-freedom when combined with
> SERIALIZABLE.
> > If
> > > > we
> > > > > > have
> > > > > > > > good deadlock detector and nice administrative capabilities,
> > this
> > > > > would
> > > > > > > not
> > > > > > > > be a problem for us.
> > > > > > >
> > > > > > >
> > > > > > > Hm... The advantage of Optimistic Serialiazable mode is
> actually
> > > > > > lock-free
> > > > > > > transactions. The deadlock is impossible in this case.
I doubt
> > any
> > > > > > deadlock
> > > > > > > detector would match the performance advantage we get from
> > > lock-free
> > > > > > > transactions.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > 2) Behavior of reads could be controlled through "with"
> facade:
> > > > > > > > V val1 = cache.get(key1);                 // Shared
lock or
> no
> > > lock
> > > > > > > > V val2 = cache.withForUpdate().get(key2); // Exclusive
lock
> > > > > > > >
> > > > > > >
> > > > > > > Don't like the API. We are not trying to abandon the data
grid
> > > > use-case
> > > > > > or
> > > > > > > API, we are trying to add the database use case.
> > > > > > >
> > > > > > >
> > > > > > > > 3) REPEATABLE_READ - throw exception in case of write-write
> > > > conflict
> > > > > > > >
> > > > > > >
> > > > > > > Well, I would like to preserve the PESSIMISTIC mode. I
find it
> > more
> > > > > > > convenient than the "withForUpdate" API. It almost seems
like
> you
> > > are
> > > > > > > trying to force the pendulum too far in the opposite direction.
> > > > > > >
> > > > > > >
> > > > > > > > 4) SERIALIZABLE - throw exception in case of write-write
and
> > > > > write-read
> > > > > > > > confilct (this is how our OPTIMISTIC/SERIALZABLE works
now,
> but
> > > it
> > > > > > > doesn't
> > > > > > > > support predicates)
> > > > > > > >
> > > > > > >
> > > > > > > So, no change here? Good :)
> > > > > > >
> > > > > > >
> > > > > > > > 5) Add READ_ONLY isolation mode where updates will
not be
> > allowed
> > > > at
> > > > > > all.
> > > > > > > > Such transacrtons would be able to bypass some Ignite
> internals
> > > to
> > > > > > > achieve
> > > > > > > > greater performance, what could be valuable for mostly-read
> use
> > > > cases
> > > > > > > (e.g.
> > > > > > > > OLAP).
> > > > > > > >
> > > > > > >
> > > > > > > Love the idea. We have already seen many use cases that
could
> > > benefit
> > > > > > from
> > > > > > > it.
> > > > > > > How hard is it to implement?
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Thoughts?
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message