ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: distributed transaction of non-single coordinator
Date Tue, 14 Mar 2017 15:55:23 GMT
What is Orchestrator for you? Is it a thing from Microsoft or your custom
in-house software?

Sergi

2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV <alkuznetsov.sb@gmail.com>:

> Fine. Let's say we've got multiple servers which fulfills custom logic.
> This servers compound oriented graph (BPMN process) which controlled by
> Orchestrator.
> For instance, *server1  *creates *variable A *with value 1, persists it to
> IGNITE cache and creates *variable B *and sends it to* server2. *The
> latests receives *variable B*, do some logic with it and stores to IGNITE.
> All the work made by both servers must be fulfilled in *one* transaction.
> Because we need all information done, or nothing(rollbacked). The scenario
> is managed by orchestrator.
>
> вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin <sergi.vladykin@gmail.com>:
>
> > Ok, it is not a business case, it is your wrong solution for it.
> > Lets try again, what is the business case?
> >
> > Sergi
> >
> > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV <alkuznetsov.sb@gmail.com>:
> >
> > > The case is the following, One starts transaction in one node, and
> commit
> > > this transaction in another jvm node(or rollback it remotely).
> > >
> > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <sergi.vladykin@gmail.com
> >:
> > >
> > > > Because even if you make it work for some simplistic scenario, get
> > ready
> > > to
> > > > write many fault tolerance tests and make sure that you TXs work
> > > gracefully
> > > > in all modes in case of crashes. Also make sure that we do not have
> any
> > > > performance drops after all your changes in existing benchmarks. All
> in
> > > all
> > > > I don't believe these conditions will be met and your contribution
> will
> > > be
> > > > accepted.
> > > >
> > > > Better solution to what problem? Sending TX to another node? The
> > problem
> > > > statement itself is already wrong. What business case you are trying
> to
> > > > solve? I'm sure everything you need can be done in a much more simple
> > and
> > > > efficient way at the application level.
> > > >
> > > > Sergi
> > > >
> > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov.sb@gmail.com
> > >:
> > > >
> > > > > Why wrong ? You know the better solution?
> > > > >
> > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > sergi.vladykin@gmail.com
> > > >:
> > > > >
> > > > > > Just serializing TX object and deserializing it on another node
> is
> > > > > > meaningless, because other nodes participating in the TX have
to
> > know
> > > > > about
> > > > > > the new coordinator. This will require protocol changes, we
> > > definitely
> > > > > will
> > > > > > have fault tolerance and performance issues. IMO the whole idea
> is
> > > > wrong
> > > > > > and it makes no sense to waste time on it.
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov.sb@gmail.com
> > > > >:
> > > > > >
> > > > > > > IgniteTransactionState implememntation contains IgniteTxEntry's
> > > which
> > > > > is
> > > > > > > supposed to be transferable
> > > > > > >
> > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org
> > > > > >:
> > > > > > >
> > > > > > > > It sounds a little scary to me that we are passing
> transaction
> > > > > objects
> > > > > > > > around. Such object may contain all sorts of Ignite
context.
> If
> > > > some
> > > > > > data
> > > > > > > > needs to be passed across, we should create a special
> transfer
> > > > object
> > > > > > in
> > > > > > > > this case.
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV
<
> > > > > > > > alkuznetsov.sb@gmail.com
> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > well, there a couple of issues preventing transaction
> > > proceeding.
> > > > > > > > > At first, After transaction serialization and
> deserialization
> > > on
> > > > > the
> > > > > > > > remote
> > > > > > > > > server, there is no txState. So im going to put
it in
> > > > > > > > > writeExternal()\readExternal()
> > > > > > > > >
> > > > > > > > > The last one is Deserialized transaction lacks
of shared
> > cache
> > > > > > context
> > > > > > > > > field at TransactionProxyImpl. Perhaps, it must
be injected
> > by
> > > > > > > > > GridResourceProcessor ?
> > > > > > > > >
> > > > > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV
<
> > > > > > > alkuznetsov.sb@gmail.com
> > > > > > > > >:
> > > > > > > > >
> > > > > > > > > > while starting and continuing transaction
in different
> jvms
> > > in
> > > > > run
> > > > > > > into
> > > > > > > > > > serialization exception in writeExternalMeta
:
> > > > > > > > > >
> > > > > > > > > > @Override public void writeExternal(ObjectOutput
out)
> > throws
> > > > > > > > IOException
> > > > > > > > > {
> > > > > > > > > >     writeExternalMeta(out);
> > > > > > > > > >
> > > > > > > > > > some meta is cannot be serialized.
> > > > > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey
Goncharuk <
> > > > > > > > > alexey.goncharuk@gmail.com
> > > > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > Aleksey,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I think I am starting to get what you want,
but I have a
> > few
> > > > > > > concerns:
> > > > > > > > > >  - What is the API for the proposed change?
In your test,
> > you
> > > > > pass
> > > > > > an
> > > > > > > > > > instance of transaction created on ignite(0)
to the
> ignite
> > > > > instance
> > > > > > > > > > ignite(1). This is obviously not possible
in a truly
> > > > distributed
> > > > > > > > > > (multi-jvm) environment.
> > > > > > > > > > - How will you synchronize cache update
actions and
> > > transaction
> > > > > > > commit?
> > > > > > > > > > Say, you have one node that decided to commit,
but
> another
> > > node
> > > > > is
> > > > > > > > still
> > > > > > > > > > writing within this transaction. How do
you make sure
> that
> > > two
> > > > > > nodes
> > > > > > > > will
> > > > > > > > > > not call commit() and rollback() simultaneously?
> > > > > > > > > >  - How do you make sure that either commit()
or
> rollback()
> > is
> > > > > > called
> > > > > > > if
> > > > > > > > > an
> > > > > > > > > > originator failed?
> > > > > > > > > >
> > > > > > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий
Рябов <
> > > > somefireone@gmail.com
> > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Alexey Goncharuk, heh, my initial understanding
was
> that
> > > > > > > transferring
> > > > > > > > > of
> > > > > > > > > > tx
> > > > > > > > > > > ownership from one node to another
will be happened
> > > > > automatically
> > > > > > > > when
> > > > > > > > > > > originating node is gone down.
> > > > > > > > > > >
> > > > > > > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY
KUZNETSOV <
> > > > > > > > alkuznetsov.sb@gmail.com
> > > > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Im aiming to span transaction
on multiple threads,
> > nodes,
> > > > > > > > jvms(soon).
> > > > > > > > > > So
> > > > > > > > > > > > every node is able to rollback,
or commit common
> > > > > transaction.It
> > > > > > > > > turned
> > > > > > > > > > > up i
> > > > > > > > > > > > need to transfer tx between nodes
in order to commit
> > > > > > transaction
> > > > > > > in
> > > > > > > > > > > > different node(in the same jvm).
> > > > > > > > > > > >
> > > > > > > > > > > > пт, 10 мар. 2017 г. в 15:20,
Alexey Goncharuk <
> > > > > > > > > > > alexey.goncharuk@gmail.com
> > > > > > > > > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > > Aleksey,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Do you mean that you want
a concept of transferring
> > of
> > > tx
> > > > > > > > ownership
> > > > > > > > > > > from
> > > > > > > > > > > > > one node to another? My initial
understanding was
> > that
> > > > you
> > > > > > want
> > > > > > > > to
> > > > > > > > > be
> > > > > > > > > > > > able
> > > > > > > > > > > > > to update keys in a transaction
from multiple
> threads
> > > in
> > > > > > > > parallel.
> > > > > > > > > > > > >
> > > > > > > > > > > > > --AG
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2017-03-10 15:01 GMT+03:00
ALEKSEY KUZNETSOV <
> > > > > > > > > > alkuznetsov.sb@gmail.com
> > > > > > > > > > > >:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Well. Consider transaction
started in one node,
> and
> > > > > > continued
> > > > > > > > in
> > > > > > > > > > > > another
> > > > > > > > > > > > > > one.
> > > > > > > > > > > > > > The following test describes
my idea:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Ignite ignite1 = ignite(0);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > IgniteTransactions transactions
=
> > > > ignite1.transactions();
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > IgniteCache<String,
Integer> cache =
> > > > > > > ignite1.getOrCreateCache("
> > > > > > > > > > > > > > testCache");
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Transaction tx = transactions.txStart(
> concurrency,
> > > > > > > isolation);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > cache.put("key1", 1);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > cache.put("key2", 2);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > tx.stop();
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > IgniteInternalFuture<Boolean>
fut =
> > > > > > GridTestUtils.runAsync(()
> > > > > > > > ->
> > > > > > > > > {
> > > > > > > > > > > > > >     IgniteTransactions
ts =
> > ignite(1).transactions();
> > > > > > > > > > > > > >     Assert.assertNull(ts.tx());
> > > > > > > > > > > > > >     Assert.assertEquals(
> TransactionState.STOPPED,
> > > > > > > tx.state());
> > > > > > > > > > > > > >     ts.txStart(tx);
> > > > > > > > > > > > > >     Assert.assertEquals(TransactionState.ACTIVE,
> > > > > > > tx.state());
> > > > > > > > > > > > > >     cache.put("key3",
3);
> > > > > > > > > > > > > >     Assert.assertTrue(cache.remove("key2"));
> > > > > > > > > > > > > >     tx.commit();
> > > > > > > > > > > > > >     return true;
> > > > > > > > > > > > > > });
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > fut.get();
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Assert.assertEquals(TransactionState.COMMITTED,
> > > > > > tx.state());
> > > > > > > > > > > > > > Assert.assertEquals((long)1,
> > > (long)cache.get("key1"));
> > > > > > > > > > > > > > Assert.assertEquals((long)3,
> > > (long)cache.get("key3"));
> > > > > > > > > > > > > > Assert.assertFalse(cache.containsKey("key2"));
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In method *ts.txStart(...)*
we just rebind *tx*
> to
> > > > > current
> > > > > > > > > thread:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > public void txStart(Transaction
tx) {
> > > > > > > > > > > > > >     TransactionProxyImpl
transactionProxy =
> > > > > > > > > > (TransactionProxyImpl)tx;
> > > > > > > > > > > > > >     cctx.tm().reopenTx(transactionProxy.tx());
> > > > > > > > > > > > > >     transactionProxy.bindToCurrentThread();
> > > > > > > > > > > > > > }
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In method *reopenTx*
we alter *threadMap* so that
> > it
> > > > > binds
> > > > > > > > > > > transaction
> > > > > > > > > > > > > > to current thread.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > How do u think about
it ?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > вт, 7 мар. 2017
г. в 22:38, Denis Magda <
> > > > > dmagda@apache.org
> > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi Alexey,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Please share the
rational behind this and the
> > > > thoughts,
> > > > > > > > design
> > > > > > > > > > > ideas
> > > > > > > > > > > > > you
> > > > > > > > > > > > > > > have in mind.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > —
> > > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Mar 7,
2017, at 3:19 AM, ALEKSEY
> KUZNETSOV <
> > > > > > > > > > > > > > alkuznetsov.sb@gmail.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi all! Im
designing distributed transaction
> > > which
> > > > > can
> > > > > > be
> > > > > > > > > > started
> > > > > > > > > > > > at
> > > > > > > > > > > > > > one
> > > > > > > > > > > > > > > > node, and
continued at other one. Has anybody
> > > > > thoughts
> > > > > > on
> > > > > > > > it
> > > > > > > > > ?
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *Kuznetsov
Aleksey*
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > >
> > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > >
> > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > *Best Regards,*
> > > > > > > > > >
> > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > *Best Regards,*
> > > > > > > > >
> > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > >
> > > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > *Best Regards,*
> > > > > > >
> > > > > > > *Kuznetsov Aleksey*
> > > > > > >
> > > > > >
> > > > > --
> > > > >
> > > > > *Best Regards,*
> > > > >
> > > > > *Kuznetsov Aleksey*
> > > > >
> > > >
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> >
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message