ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Pavlov <dpav...@apache.org>
Subject Re: Thin client: transactions support
Date Thu, 28 Mar 2019 11:33:44 GMT
Hi,

I've added permissions to account plehanov.alex

Recently Infra integrated Apache LDAP with confluence, so it is possible to
login using Apache credentials. Probably we can ask infra if extra
permissions to edit pages should be added for committers.

Sincerely,
Dmitriy Pavlov

ср, 27 мар. 2019 г. в 13:37, Alex Plehanov <plehanov.alex@gmail.com>:

> Vladimir,
>
> About current tx: ok, then we don't need tx() method in the interface at
> all (the same cached transaction info user can store by himself).
>
> About decoupling transactions from threads on the server side: for now, we
> can start with thread-per-connection approach (we only can support one
> active transaction per connection, see below, so we need one additional
> dedicated thread for each connection with active transaction), and later
> change server-side internals to process client transactions in any server
> thread (not dedicated to this connection). This change will not affect the
> thin client protocol, it only affects the server side.
> In any case, we can't support concurrent transactions per connection on
> the client side without fundamental changes to the current protocol (cache
> operation doesn't bound to transaction or thread and the server doesn't
> know which thread on the client side do this cache operation). In my
> opinion, if a user wants to use concurrent transactions, he must use
> different connections from a connection pool.
>
> About semantics of suspend/resume on the client-side: it's absolutely
> different than server-side semantics (we don't need to do suspend/resume to
> pass transaction between threads on the client-side), but can't be
> implemented efficiently without implemented suspend/resume on server-side.
>
> Can anyone give me permissions to create IEP on Apache wiki?
>
> ср, 27 мар. 2019 г. в 11:59, Vladimir Ozerov <vozerov@gridgain.com>:
>
> > Hi Alex,
> >
> > My comments was only about the protocol. Getting current info about
> > transaction should be handled by the client itself. It is not protocl's
> > concern. Same about other APIs and behavior in case another transaction
> is
> > attempted from the same thread.
> >
> > Putting protocol aside, transaction support is complicated matter. I
> would
> > propose to route through IEP and wide community discussion. We need to
> > review API and semantics very carefully, taking SUSPEND/RESUME in count.
> > Also I do not see how we support client transactions efficiently without
> > decoupling transactions from threads on the server side first. Because
> > without it you will need a dedicated server thread for every client's
> > transaction which is slow and may even crash the server.
> >
> > Vladimir.
> >
> > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov <plehanov.alex@gmail.com>
> > wrote:
> >
> > > Vladimir, what if we want to get current transaction info (tx()
> method)?
> > >
> > > Does close() method mapped to TX_END(rollback)?
> > >
> > > For example, this code:
> > >
> > > try(tx = txStart()) {
> > >     tx.commit();
> > > }
> > >
> > > Will produce:
> > > TX_START
> > > TX_END(commit)
> > > TX_END(rollback)
> > >
> > > Am I understand you right?
> > >
> > > About xid. There is yet another proposal. Use some unique per
> connection
> > id
> > > (integer, simple counter) for identifying the transaction on
> > > commit/rollback message. The client gets this id from the server with
> > > transaction info and sends it back to the server when trying to
> > > commit/rollback transaction. This id is not shown to users. But also we
> > can
> > > pass from server to client real transaction id (xid) with transaction
> > info
> > > for diagnostic purposes.
> > >
> > > And one more question: what should we do if the client starts a new
> > > transaction without ending the old one? Should we end the old
> transaction
> > > implicitly (rollback) or throw an exception to the client? In my
> opinion,
> > > the first option is better. For example, if we got a previously used
> > > connection from the connection pool, we should not worry about any
> > > uncompleted transaction started by the previous user of this
> connection.
> > >
> > > ср, 27 мар. 2019 г. в 11:02, Vladimir Ozerov <vozerov@gridgain.com>:
> > >
> > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not support them yet, and
> > > adding
> > > > them in future should not conflict with simple START/END
> > infrastructure.
> > > >
> > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Hi Alex,
> > > > >
> > > > > I am not sure we need 5 commands. Wouldn't it be enough to have
> only
> > > two?
> > > > >
> > > > > START - accepts optional parameters, returns transaction info
> > > > > END - provides commit flag, returns void
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov <
> > plehanov.alex@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Sergey, yes, the close is something like silent rollback. But
we
> can
> > > > >> also implement this on the client side, just using rollback and
> > > ignoring
> > > > >> errors in the response.
> > > > >>
> > > > >> ср, 27 мар. 2019 г. в 00:04, Sergey Kozlov <skozlov@gridgain.com
> >:
> > > > >>
> > > > >> > Nikolay
> > > > >> >
> > > > >> > Am I correctly understand you points:
> > > > >> >
> > > > >> >    - close: rollback
> > > > >> >    - commit, close: do nothing
> > > > >> >    - rollback, close: do what? (I suppose nothing)
> > > > >> >
> > > > >> > Also you assume that after commit/rollback we may need to
free
> > some
> > > > >> > resources on server node(s)or just do on client started
TX?
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex Plehanov <
> > > > plehanov.alex@gmail.com
> > > > >> >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Sergey, we have the close() method in the thick client,
it's
> > > > behavior
> > > > >> is
> > > > >> > > slightly different than rollback() method (it should
rollback
> if
> > > the
> > > > >> > > transaction is not committed and do nothing if the
transaction
> > is
> > > > >> already
> > > > >> > > committed). I think we should support try-with-resource
> > semantics
> > > in
> > > > >> the
> > > > >> > > thin client and OP_TX_CLOSE will be useful here.
> > > > >> > >
> > > > >> > > Nikolay, suspend/resume didn't work yet for pessimistic
> > > > transactions.
> > > > >> > Also,
> > > > >> > > the main goal of suspend/resume operations is to support
> > > transaction
> > > > >> > > passing between threads. In the thin client, the transaction
> is
> > > > bound
> > > > >> to
> > > > >> > > the client connection, not client thread. I think passing
a
> > > > >> transaction
> > > > >> > > between different client connections is not a very
useful
> case.
> > > > >> > >
> > > > >> > > вт, 26 мар. 2019 г. в 22:17, Nikolay Izhikov
<
> > nizhikov@apache.org
> > > >:
> > > > >> > >
> > > > >> > > > Hello, Alex.
> > > > >> > > >
> > > > >> > > > We also have suspend and resume operations.
> > > > >> > > > I think we should support them
> > > > >> > > >
> > > > >> > > > вт, 26 марта 2019 г., 22:07 Sergey Kozlov
<
> > skozlov@gridgain.com
> > > >:
> > > > >> > > >
> > > > >> > > > > Hi
> > > > >> > > > >
> > > > >> > > > > Looks like I missed something but why we
need OP_TX_CLOSE
> > > > >> operation?
> > > > >> > > > >
> > > > >> > > > > Also I suggest to reserve a code for SAVEPOINT
operation
> > which
> > > > >> very
> > > > >> > > > useful
> > > > >> > > > > to understand where transaction has been
rolled back
> > > > >> > > > >
> > > > >> > > > > On Tue, Mar 26, 2019 at 6:07 PM Alex Plehanov
<
> > > > >> > plehanov.alex@gmail.com
> > > > >> > > >
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hello Igniters!
> > > > >> > > > > >
> > > > >> > > > > > I want to pick up the ticket IGNITE-7369
and add
> > > transactions
> > > > >> > support
> > > > >> > > > to
> > > > >> > > > > > our thin client implementation.
> > > > >> > > > > > I've looked at our current implementation
and have some
> > > > >> proposals
> > > > >> > to
> > > > >> > > > > > support transactions:
> > > > >> > > > > >
> > > > >> > > > > > Add new operations to thin client protocol:
> > > > >> > > > > >
> > > > >> > > > > >     OP_TX_GET, 4000, Get current transaction
for client
> > > > >> connection
> > > > >> > > > > >     OP_TX_START, 4001, Start a new transaction
> > > > >> > > > > >     OP_TX_COMMIT, 4002, Commit transaction
> > > > >> > > > > >     OP_TX_ROLLBACK, 4003, Rollback transaction
> > > > >> > > > > >     OP_TX_CLOSE, 4004, Close transaction
> > > > >> > > > > >
> > > > >> > > > > > From the client side (java) new interfaces
will be
> added:
> > > > >> > > > > >
> > > > >> > > > > > public interface ClientTransactions
{
> > > > >> > > > > >     public ClientTransaction txStart();
> > > > >> > > > > >     public ClientTransaction
> > txStart(TransactionConcurrency
> > > > >> > > > concurrency,
> > > > >> > > > > > TransactionIsolation isolation);
> > > > >> > > > > >     public ClientTransaction
> > txStart(TransactionConcurrency
> > > > >> > > > concurrency,
> > > > >> > > > > > TransactionIsolation isolation, long
timeout, int
> txSize);
> > > > >> > > > > >     public ClientTransaction tx(); //
Get current
> > connection
> > > > >> > > > transaction
> > > > >> > > > > >     public ClientTransactions withLabel(String
lb);
> > > > >> > > > > > }
> > > > >> > > > > >
> > > > >> > > > > > public interface ClientTransaction extends
> AutoCloseable {
> > > > >> > > > > >     public IgniteUuid xid(); // Do we
need it?
> > > > >> > > > > >     public TransactionIsolation isolation();
> > > > >> > > > > >     public TransactionConcurrency concurrency();
> > > > >> > > > > >     public long timeout();
> > > > >> > > > > >     public String label();
> > > > >> > > > > >
> > > > >> > > > > >     public void commit();
> > > > >> > > > > >     public void rollback();
> > > > >> > > > > >     public void close();
> > > > >> > > > > > }
> > > > >> > > > > >
> > > > >> > > > > > From the server side, I think as a first
step (while
> > > > >> transactions
> > > > >> > > > > > suspend/resume is not fully implemented)
we can use the
> > same
> > > > >> > approach
> > > > >> > > > as
> > > > >> > > > > > for JDBC: add a new worker to each ClientRequestHandler
> > and
> > > > >> process
> > > > >> > > > > > requests by this worker if the transaction
is started
> > > > >> explicitly.
> > > > >> > > > > > ClientRequestHandler is bound to client
connection, so
> > there
> > > > >> will
> > > > >> > be
> > > > >> > > > 1:1
> > > > >> > > > > > relation between client connection and
thread, which
> > process
> > > > >> > > operations
> > > > >> > > > > in
> > > > >> > > > > > a transaction.
> > > > >> > > > > >
> > > > >> > > > > > Also, there is a couple of issues I
want to discuss:
> > > > >> > > > > >
> > > > >> > > > > > We have overloaded method txStart with
a different set
> of
> > > > >> > arguments.
> > > > >> > > > Some
> > > > >> > > > > > of the arguments may be missing. To
pass arguments with
> > > > >> OP_TX_START
> > > > >> > > > > > operation we have the next options:
> > > > >> > > > > >  * Serialize full set of arguments and
use some value
> for
> > > > >> missing
> > > > >> > > > > > arguments. For example -1 for int/long
types and null
> for
> > > > string
> > > > >> > > type.
> > > > >> > > > We
> > > > >> > > > > > can't use 0 for int/long types since
0 it's a valid
> value
> > > for
> > > > >> > > > > concurrency,
> > > > >> > > > > > isolation and timeout arguments.
> > > > >> > > > > >  * Serialize arguments as a collection
of property-value
> > > pairs
> > > > >> > (like
> > > > >> > > > it's
> > > > >> > > > > > implemented now for CacheConfiguration).
In this case
> only
> > > > >> > explicitly
> > > > >> > > > > > provided arguments will be serialized.
> > > > >> > > > > > Which way is better? The simplest solution
is to use the
> > > first
> > > > >> > option
> > > > >> > > > > and I
> > > > >> > > > > > want to use it if there were no objections.
> > > > >> > > > > >
> > > > >> > > > > > Do we need transaction id (xid) on the
client side?
> > > > >> > > > > > If yes, we can pass xid along with OP_TX_COMMIT,
> > > > OP_TX_ROLLBACK,
> > > > >> > > > > > OP_TX_CLOSE operations back to the server
and do
> > additional
> > > > >> check
> > > > >> > on
> > > > >> > > > the
> > > > >> > > > > > server side (current transaction id
for connection ==
> > > > >> transaction
> > > > >> > id
> > > > >> > > > > passed
> > > > >> > > > > > from client side). This, perhaps, will
protect clients
> > > against
> > > > >> some
> > > > >> > > > > errors
> > > > >> > > > > > (for example when client try to commit
outdated
> > > transaction).
> > > > >> But
> > > > >> > > > > > currently, we don't have data type IgniteUuid
in thin
> > client
> > > > >> > > protocol.
> > > > >> > > > Do
> > > > >> > > > > > we need to add it too?
> > > > >> > > > > > Also, we can pass xid as a string just
to inform the
> > client
> > > > and
> > > > >> do
> > > > >> > > not
> > > > >> > > > > pass
> > > > >> > > > > > it back to the server with commit/rollback
operation.
> > > > >> > > > > > Or not to pass xid at all (.NET thick
client works this
> > way
> > > as
> > > > >> far
> > > > >> > > as I
> > > > >> > > > > > know).
> > > > >> > > > > >
> > > > >> > > > > > What do you think?
> > > > >> > > > > >
> > > > >> > > > > > ср, 7 мар. 2018 г. в 16:22, Vladimir
Ozerov <
> > > > >> vozerov@gridgain.com
> > > > >> > >:
> > > > >> > > > > >
> > > > >> > > > > > > We already have transactions support
in JDBC driver in
> > TX
> > > > SQL
> > > > >> > > branch
> > > > >> > > > > > > (ignite-4191). Currently it is
implemented through
> > > separate
> > > > >> > thread,
> > > > >> > > > > which
> > > > >> > > > > > > is not that efficient. Ideally
we need to finish
> > > decoupling
> > > > >> > > > > transactions
> > > > >> > > > > > > from threads. But alternatively
we can change the
> logic
> > on
> > > > >> how we
> > > > >> > > > > assign
> > > > >> > > > > > > thread ID to specific transaction
and "impersonate"
> thin
> > > > >> client
> > > > >> > > > worker
> > > > >> > > > > > > threads when serving requests from
multiple users.
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > > On Tue, Mar 6, 2018 at 10:01 PM,
Denis Magda <
> > > > >> dmagda@apache.org>
> > > > >> > > > > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > > Here is an original discussion
with a reference to
> the
> > > > JIRA
> > > > >> > > ticket:
> > > > >> > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > >> > > > > > > >
> > > > com/Re-Transaction-operations-using-the-Ignite-Thin-Client-
> > > > >> > > > > > > > Protocol-td25914.html
> > > > >> > > > > > > >
> > > > >> > > > > > > > --
> > > > >> > > > > > > > Denis
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Tue, Mar 6, 2018 at 9:18
AM, Dmitriy Setrakyan <
> > > > >> > > > > > dsetrakyan@apache.org
> > > > >> > > > > > > >
> > > > >> > > > > > > > wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > Hi Dmitriy. I don't think
we have a design
> proposal
> > > for
> > > > >> > > > transaction
> > > > >> > > > > > > > support
> > > > >> > > > > > > > > in thin clients. Do you
mind taking this
> initiative
> > > and
> > > > >> > > creating
> > > > >> > > > an
> > > > >> > > > > > IEP
> > > > >> > > > > > > > on
> > > > >> > > > > > > > > Wiki?
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > D.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > On Tue, Mar 6, 2018 at
8:46 AM, Dmitriy
> Govorukhin <
> > > > >> > > > > > > > > dmitriy.govorukhin@gmail.com>
wrote:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > > Hi, Igniters.
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > I've seen a lot
of discussions about thin client
> > and
> > > > >> binary
> > > > >> > > > > > protocol,
> > > > >> > > > > > > > > but I
> > > > >> > > > > > > > > > did not hear anything
about transactions
> support.
> > Do
> > > > we
> > > > >> > have
> > > > >> > > > some
> > > > >> > > > > > > draft
> > > > >> > > > > > > > > for
> > > > >> > > > > > > > > > this purpose?
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > As I understand
we have several problems:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > >    - thread and
transaction have hard related
> (we
> > > use
> > > > >> > > > > thread-local
> > > > >> > > > > > > > > variable
> > > > >> > > > > > > > > >    and thread name)
> > > > >> > > > > > > > > >    - we can process
only one transaction at the
> > same
> > > > >> time
> > > > >> > in
> > > > >> > > > one
> > > > >> > > > > > > thread
> > > > >> > > > > > > > > (it
> > > > >> > > > > > > > > >    mean we need
hold thread per client. If
> connect
> > > 100
> > > > >> thin
> > > > >> > > > > clients
> > > > >> > > > > > > to
> > > > >> > > > > > > > 1
> > > > >> > > > > > > > > >    server node,
then need to hold 100 thread on
> > the
> > > > >> server
> > > > >> > > > side)
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Let's discuss how
we can implement transactions
> > for
> > > > the
> > > > >> > thin
> > > > >> > > > > > client.
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > > Sergey Kozlov
> > > > >> > > > > GridGain Systems
> > > > >> > > > > www.gridgain.com
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > Sergey Kozlov
> > > > >> > GridGain Systems
> > > > >> > www.gridgain.com
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message