ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Plehanov <plehanov.a...@gmail.com>
Subject Re: Thin client: transactions support
Date Tue, 02 Apr 2019 08:29:38 GMT
Pavel,

AFAIK usually, such limitations handled by using connection pools. If a
thread needs to start a transaction, it acquires a connection from the pool
and releases the connection after the transaction end.

вт, 2 апр. 2019 г. в 09:55, Pavel Tupitsyn <ptupitsyn@apache.org>:

> Alex,
>
> > now we can only support one active transaction per connection
>
> I totally understand server-side and protocol limitations that are causing
> this.
> But I have no idea how to support this in .NET Thin Client, for example.
>
> It is thread-safe and can handle multiple async operations in parallel.
> But with TX support we have to somehow switch to single-threaded mode to
> avoid unexpected effects.
>
> Any ideas?
>
>
> On Mon, Apr 1, 2019 at 6:38 PM Alex Plehanov <plehanov.alex@gmail.com>
> wrote:
>
> > Dmitriy, thank you!
> >
> > Guys, I've created the IEP [1] on wiki, please have a look.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> >
> >
> > чт, 28 мар. 2019 г. в 14:33, Dmitriy Pavlov <dpavlov@apache.org>:
> >
> > > Hi,
> > >
> > > I've added permissions to account plehanov.alex
> > >
> > > Recently Infra integrated Apache LDAP with confluence, so it is
> possible
> > to
> > > login using Apache credentials. Probably we can ask infra if extra
> > > permissions to edit pages should be added for committers.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > ср, 27 мар. 2019 г. в 13:37, Alex Plehanov <plehanov.alex@gmail.com>:
> > >
> > > > Vladimir,
> > > >
> > > > About current tx: ok, then we don't need tx() method in the interface
> > at
> > > > all (the same cached transaction info user can store by himself).
> > > >
> > > > About decoupling transactions from threads on the server side: for
> now,
> > > we
> > > > can start with thread-per-connection approach (we only can support
> one
> > > > active transaction per connection, see below, so we need one
> additional
> > > > dedicated thread for each connection with active transaction), and
> > later
> > > > change server-side internals to process client transactions in any
> > server
> > > > thread (not dedicated to this connection). This change will not
> affect
> > > the
> > > > thin client protocol, it only affects the server side.
> > > > In any case, we can't support concurrent transactions per connection
> on
> > > > the client side without fundamental changes to the current protocol
> > > (cache
> > > > operation doesn't bound to transaction or thread and the server
> doesn't
> > > > know which thread on the client side do this cache operation). In my
> > > > opinion, if a user wants to use concurrent transactions, he must use
> > > > different connections from a connection pool.
> > > >
> > > > About semantics of suspend/resume on the client-side: it's absolutely
> > > > different than server-side semantics (we don't need to do
> > suspend/resume
> > > to
> > > > pass transaction between threads on the client-side), but can't be
> > > > implemented efficiently without implemented suspend/resume on
> > > server-side.
> > > >
> > > > Can anyone give me permissions to create IEP on Apache wiki?
> > > >
> > > > ср, 27 мар. 2019 г. в 11:59, Vladimir Ozerov <vozerov@gridgain.com>:
> > > >
> > > > > Hi Alex,
> > > > >
> > > > > My comments was only about the protocol. Getting current info about
> > > > > transaction should be handled by the client itself. It is not
> > protocl's
> > > > > concern. Same about other APIs and behavior in case another
> > transaction
> > > > is
> > > > > attempted from the same thread.
> > > > >
> > > > > Putting protocol aside, transaction support is complicated matter.
> I
> > > > would
> > > > > propose to route through IEP and wide community discussion. We need
> > to
> > > > > review API and semantics very carefully, taking SUSPEND/RESUME in
> > > count.
> > > > > Also I do not see how we support client transactions efficiently
> > > without
> > > > > decoupling transactions from threads on the server side first.
> > Because
> > > > > without it you will need a dedicated server thread for every
> client's
> > > > > transaction which is slow and may even crash the server.
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov <
> > > plehanov.alex@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Vladimir, what if we want to get current transaction info (tx()
> > > > method)?
> > > > > >
> > > > > > Does close() method mapped to TX_END(rollback)?
> > > > > >
> > > > > > For example, this code:
> > > > > >
> > > > > > try(tx = txStart()) {
> > > > > >     tx.commit();
> > > > > > }
> > > > > >
> > > > > > Will produce:
> > > > > > TX_START
> > > > > > TX_END(commit)
> > > > > > TX_END(rollback)
> > > > > >
> > > > > > Am I understand you right?
> > > > > >
> > > > > > About xid. There is yet another proposal. Use some unique per
> > > > connection
> > > > > id
> > > > > > (integer, simple counter) for identifying the transaction on
> > > > > > commit/rollback message. The client gets this id from the server
> > with
> > > > > > transaction info and sends it back to the server when trying
to
> > > > > > commit/rollback transaction. This id is not shown to users.
But
> > also
> > > we
> > > > > can
> > > > > > pass from server to client real transaction id (xid) with
> > transaction
> > > > > info
> > > > > > for diagnostic purposes.
> > > > > >
> > > > > > And one more question: what should we do if the client starts
a
> new
> > > > > > transaction without ending the old one? Should we end the old
> > > > transaction
> > > > > > implicitly (rollback) or throw an exception to the client? In
my
> > > > opinion,
> > > > > > the first option is better. For example, if we got a previously
> > used
> > > > > > connection from the connection pool, we should not worry about
> any
> > > > > > uncompleted transaction started by the previous user of this
> > > > connection.
> > > > > >
> > > > > > ср, 27 мар. 2019 г. в 11:02, Vladimir Ozerov <
> vozerov@gridgain.com
> > >:
> > > > > >
> > > > > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not support
them
> yet,
> > > and
> > > > > > adding
> > > > > > > them in future should not conflict with simple START/END
> > > > > infrastructure.
> > > > > > >
> > > > > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov <
> > > > vozerov@gridgain.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Alex,
> > > > > > > >
> > > > > > > > I am not sure we need 5 commands. Wouldn't it be enough
to
> have
> > > > only
> > > > > > two?
> > > > > > > >
> > > > > > > > START - accepts optional parameters, returns transaction
info
> > > > > > > > END - provides commit flag, returns void
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov <
> > > > > plehanov.alex@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Sergey, yes, the close is something like silent
rollback.
> But
> > we
> > > > can
> > > > > > > >> also implement this on the client side, just using
rollback
> > and
> > > > > > ignoring
> > > > > > > >> errors in the response.
> > > > > > > >>
> > > > > > > >> ср, 27 мар. 2019 г. в 00:04, Sergey Kozlov
<
> > > skozlov@gridgain.com
> > > > >:
> > > > > > > >>
> > > > > > > >> > Nikolay
> > > > > > > >> >
> > > > > > > >> > Am I correctly understand you points:
> > > > > > > >> >
> > > > > > > >> >    - close: rollback
> > > > > > > >> >    - commit, close: do nothing
> > > > > > > >> >    - rollback, close: do what? (I suppose
nothing)
> > > > > > > >> >
> > > > > > > >> > Also you assume that after commit/rollback
we may need to
> > free
> > > > > some
> > > > > > > >> > resources on server node(s)or just do on
client started
> TX?
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex Plehanov
<
> > > > > > > plehanov.alex@gmail.com
> > > > > > > >> >
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> > > Sergey, we have the close() method in
the thick client,
> > it's
> > > > > > > behavior
> > > > > > > >> is
> > > > > > > >> > > slightly different than rollback() method
(it should
> > > rollback
> > > > if
> > > > > > the
> > > > > > > >> > > transaction is not committed and do
nothing if the
> > > transaction
> > > > > is
> > > > > > > >> already
> > > > > > > >> > > committed). I think we should support
try-with-resource
> > > > > semantics
> > > > > > in
> > > > > > > >> the
> > > > > > > >> > > thin client and OP_TX_CLOSE will be
useful here.
> > > > > > > >> > >
> > > > > > > >> > > Nikolay, suspend/resume didn't work
yet for pessimistic
> > > > > > > transactions.
> > > > > > > >> > Also,
> > > > > > > >> > > the main goal of suspend/resume operations
is to support
> > > > > > transaction
> > > > > > > >> > > passing between threads. In the thin
client, the
> > transaction
> > > > is
> > > > > > > bound
> > > > > > > >> to
> > > > > > > >> > > the client connection, not client thread.
I think
> passing
> > a
> > > > > > > >> transaction
> > > > > > > >> > > between different client connections
is not a very
> useful
> > > > case.
> > > > > > > >> > >
> > > > > > > >> > > вт, 26 мар. 2019 г. в 22:17,
Nikolay Izhikov <
> > > > > nizhikov@apache.org
> > > > > > >:
> > > > > > > >> > >
> > > > > > > >> > > > Hello, Alex.
> > > > > > > >> > > >
> > > > > > > >> > > > We also have suspend and resume
operations.
> > > > > > > >> > > > I think we should support them
> > > > > > > >> > > >
> > > > > > > >> > > > вт, 26 марта 2019 г., 22:07
Sergey Kozlov <
> > > > > skozlov@gridgain.com
> > > > > > >:
> > > > > > > >> > > >
> > > > > > > >> > > > > Hi
> > > > > > > >> > > > >
> > > > > > > >> > > > > Looks like I missed something
but why we need
> > > OP_TX_CLOSE
> > > > > > > >> operation?
> > > > > > > >> > > > >
> > > > > > > >> > > > > Also I suggest to reserve
a code for SAVEPOINT
> > operation
> > > > > which
> > > > > > > >> very
> > > > > > > >> > > > useful
> > > > > > > >> > > > > to understand where transaction
has been rolled back
> > > > > > > >> > > > >
> > > > > > > >> > > > > On Tue, Mar 26, 2019 at 6:07
PM Alex Plehanov <
> > > > > > > >> > plehanov.alex@gmail.com
> > > > > > > >> > > >
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > >
> > > > > > > >> > > > > > Hello Igniters!
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > I want to pick up the
ticket IGNITE-7369 and add
> > > > > > transactions
> > > > > > > >> > support
> > > > > > > >> > > > to
> > > > > > > >> > > > > > our thin client implementation.
> > > > > > > >> > > > > > I've looked at our current
implementation and have
> > > some
> > > > > > > >> proposals
> > > > > > > >> > to
> > > > > > > >> > > > > > support transactions:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Add new operations to
thin client protocol:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >     OP_TX_GET, 4000,
Get current transaction for
> > > client
> > > > > > > >> connection
> > > > > > > >> > > > > >     OP_TX_START, 4001,
Start a new transaction
> > > > > > > >> > > > > >     OP_TX_COMMIT, 4002,
Commit transaction
> > > > > > > >> > > > > >     OP_TX_ROLLBACK, 4003,
Rollback transaction
> > > > > > > >> > > > > >     OP_TX_CLOSE, 4004,
Close transaction
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > From the client side
(java) new interfaces will be
> > > > added:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > public interface ClientTransactions
{
> > > > > > > >> > > > > >     public ClientTransaction
txStart();
> > > > > > > >> > > > > >     public ClientTransaction
> > > > > txStart(TransactionConcurrency
> > > > > > > >> > > > concurrency,
> > > > > > > >> > > > > > TransactionIsolation
isolation);
> > > > > > > >> > > > > >     public ClientTransaction
> > > > > txStart(TransactionConcurrency
> > > > > > > >> > > > concurrency,
> > > > > > > >> > > > > > TransactionIsolation
isolation, long timeout, int
> > > > txSize);
> > > > > > > >> > > > > >     public ClientTransaction
tx(); // Get current
> > > > > connection
> > > > > > > >> > > > transaction
> > > > > > > >> > > > > >     public ClientTransactions
withLabel(String
> lb);
> > > > > > > >> > > > > > }
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > public interface ClientTransaction
extends
> > > > AutoCloseable {
> > > > > > > >> > > > > >     public IgniteUuid
xid(); // Do we need it?
> > > > > > > >> > > > > >     public TransactionIsolation
isolation();
> > > > > > > >> > > > > >     public TransactionConcurrency
concurrency();
> > > > > > > >> > > > > >     public long timeout();
> > > > > > > >> > > > > >     public String label();
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >     public void commit();
> > > > > > > >> > > > > >     public void rollback();
> > > > > > > >> > > > > >     public void close();
> > > > > > > >> > > > > > }
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > From the server side,
I think as a first step
> (while
> > > > > > > >> transactions
> > > > > > > >> > > > > > suspend/resume is not
fully implemented) we can
> use
> > > the
> > > > > same
> > > > > > > >> > approach
> > > > > > > >> > > > as
> > > > > > > >> > > > > > for JDBC: add a new worker
to each
> > > ClientRequestHandler
> > > > > and
> > > > > > > >> process
> > > > > > > >> > > > > > requests by this worker
if the transaction is
> > started
> > > > > > > >> explicitly.
> > > > > > > >> > > > > > ClientRequestHandler
is bound to client
> connection,
> > so
> > > > > there
> > > > > > > >> will
> > > > > > > >> > be
> > > > > > > >> > > > 1:1
> > > > > > > >> > > > > > relation between client
connection and thread,
> which
> > > > > process
> > > > > > > >> > > operations
> > > > > > > >> > > > > in
> > > > > > > >> > > > > > a transaction.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Also, there is a couple
of issues I want to
> discuss:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > We have overloaded method
txStart with a different
> > set
> > > > of
> > > > > > > >> > arguments.
> > > > > > > >> > > > Some
> > > > > > > >> > > > > > of the arguments may
be missing. To pass arguments
> > > with
> > > > > > > >> OP_TX_START
> > > > > > > >> > > > > > operation we have the
next options:
> > > > > > > >> > > > > >  * Serialize full set
of arguments and use some
> > value
> > > > for
> > > > > > > >> missing
> > > > > > > >> > > > > > arguments. For example
-1 for int/long types and
> > null
> > > > for
> > > > > > > string
> > > > > > > >> > > type.
> > > > > > > >> > > > We
> > > > > > > >> > > > > > can't use 0 for int/long
types since 0 it's a
> valid
> > > > value
> > > > > > for
> > > > > > > >> > > > > concurrency,
> > > > > > > >> > > > > > isolation and timeout
arguments.
> > > > > > > >> > > > > >  * Serialize arguments
as a collection of
> > > property-value
> > > > > > pairs
> > > > > > > >> > (like
> > > > > > > >> > > > it's
> > > > > > > >> > > > > > implemented now for CacheConfiguration).
In this
> > case
> > > > only
> > > > > > > >> > explicitly
> > > > > > > >> > > > > > provided arguments will
be serialized.
> > > > > > > >> > > > > > Which way is better?
The simplest solution is to
> use
> > > the
> > > > > > first
> > > > > > > >> > option
> > > > > > > >> > > > > and I
> > > > > > > >> > > > > > want to use it if there
were no objections.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Do we need transaction
id (xid) on the client
> side?
> > > > > > > >> > > > > > If yes, we can pass xid
along with OP_TX_COMMIT,
> > > > > > > OP_TX_ROLLBACK,
> > > > > > > >> > > > > > OP_TX_CLOSE operations
back to the server and do
> > > > > additional
> > > > > > > >> check
> > > > > > > >> > on
> > > > > > > >> > > > the
> > > > > > > >> > > > > > server side (current
transaction id for connection
> > ==
> > > > > > > >> transaction
> > > > > > > >> > id
> > > > > > > >> > > > > passed
> > > > > > > >> > > > > > from client side). This,
perhaps, will protect
> > clients
> > > > > > against
> > > > > > > >> some
> > > > > > > >> > > > > errors
> > > > > > > >> > > > > > (for example when client
try to commit outdated
> > > > > > transaction).
> > > > > > > >> But
> > > > > > > >> > > > > > currently, we don't have
data type IgniteUuid in
> > thin
> > > > > client
> > > > > > > >> > > protocol.
> > > > > > > >> > > > Do
> > > > > > > >> > > > > > we need to add it too?
> > > > > > > >> > > > > > Also, we can pass xid
as a string just to inform
> the
> > > > > client
> > > > > > > and
> > > > > > > >> do
> > > > > > > >> > > not
> > > > > > > >> > > > > pass
> > > > > > > >> > > > > > it back to the server
with commit/rollback
> > operation.
> > > > > > > >> > > > > > Or not to pass xid at
all (.NET thick client works
> > > this
> > > > > way
> > > > > > as
> > > > > > > >> far
> > > > > > > >> > > as I
> > > > > > > >> > > > > > know).
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > What do you think?
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > ср, 7 мар. 2018
г. в 16:22, Vladimir Ozerov <
> > > > > > > >> vozerov@gridgain.com
> > > > > > > >> > >:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > > We already have
transactions support in JDBC
> > driver
> > > in
> > > > > TX
> > > > > > > SQL
> > > > > > > >> > > branch
> > > > > > > >> > > > > > > (ignite-4191). Currently
it is implemented
> through
> > > > > > separate
> > > > > > > >> > thread,
> > > > > > > >> > > > > which
> > > > > > > >> > > > > > > is not that efficient.
Ideally we need to finish
> > > > > > decoupling
> > > > > > > >> > > > > transactions
> > > > > > > >> > > > > > > from threads. But
alternatively we can change
> the
> > > > logic
> > > > > on
> > > > > > > >> how we
> > > > > > > >> > > > > assign
> > > > > > > >> > > > > > > thread ID to specific
transaction and
> > "impersonate"
> > > > thin
> > > > > > > >> client
> > > > > > > >> > > > worker
> > > > > > > >> > > > > > > threads when serving
requests from multiple
> users.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > On Tue, Mar 6, 2018
at 10:01 PM, Denis Magda <
> > > > > > > >> dmagda@apache.org>
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > > Here is an
original discussion with a
> reference
> > to
> > > > the
> > > > > > > JIRA
> > > > > > > >> > > ticket:
> > > > > > > >> > > > > > > >
> > http://apache-ignite-developers.2346864.n4.nabble
> > > .
> > > > > > > >> > > > > > > >
> > > > > > > com/Re-Transaction-operations-using-the-Ignite-Thin-Client-
> > > > > > > >> > > > > > > > Protocol-td25914.html
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > --
> > > > > > > >> > > > > > > > Denis
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > On Tue, Mar
6, 2018 at 9:18 AM, Dmitriy
> > Setrakyan
> > > <
> > > > > > > >> > > > > > dsetrakyan@apache.org
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > wrote:
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > > Hi Dmitriy.
I don't think we have a design
> > > > proposal
> > > > > > for
> > > > > > > >> > > > transaction
> > > > > > > >> > > > > > > > support
> > > > > > > >> > > > > > > > > in thin
clients. Do you mind taking this
> > > > initiative
> > > > > > and
> > > > > > > >> > > creating
> > > > > > > >> > > > an
> > > > > > > >> > > > > > IEP
> > > > > > > >> > > > > > > > on
> > > > > > > >> > > > > > > > > Wiki?
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > D.
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > On Tue,
Mar 6, 2018 at 8:46 AM, Dmitriy
> > > > Govorukhin <
> > > > > > > >> > > > > > > > > dmitriy.govorukhin@gmail.com>
wrote:
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > > Hi,
Igniters.
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > I've
seen a lot of discussions about thin
> > > client
> > > > > and
> > > > > > > >> binary
> > > > > > > >> > > > > > protocol,
> > > > > > > >> > > > > > > > > but I
> > > > > > > >> > > > > > > > > > did
not hear anything about transactions
> > > > support.
> > > > > Do
> > > > > > > we
> > > > > > > >> > have
> > > > > > > >> > > > some
> > > > > > > >> > > > > > > draft
> > > > > > > >> > > > > > > > > for
> > > > > > > >> > > > > > > > > > this
purpose?
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > As
I understand we have several problems:
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > 
  - thread and transaction have hard
> > related
> > > > (we
> > > > > > use
> > > > > > > >> > > > > thread-local
> > > > > > > >> > > > > > > > > variable
> > > > > > > >> > > > > > > > > > 
  and thread name)
> > > > > > > >> > > > > > > > > > 
  - we can process only one transaction
> at
> > > the
> > > > > same
> > > > > > > >> time
> > > > > > > >> > in
> > > > > > > >> > > > one
> > > > > > > >> > > > > > > thread
> > > > > > > >> > > > > > > > > (it
> > > > > > > >> > > > > > > > > > 
  mean we need hold thread per client. If
> > > > connect
> > > > > > 100
> > > > > > > >> thin
> > > > > > > >> > > > > clients
> > > > > > > >> > > > > > > to
> > > > > > > >> > > > > > > > 1
> > > > > > > >> > > > > > > > > > 
  server node, then need to hold 100
> thread
> > > on
> > > > > the
> > > > > > > >> server
> > > > > > > >> > > > side)
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > Let's
discuss how we can implement
> > > transactions
> > > > > for
> > > > > > > the
> > > > > > > >> > thin
> > > > > > > >> > > > > > client.
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > > --
> > > > > > > >> > > > > Sergey Kozlov
> > > > > > > >> > > > > GridGain Systems
> > > > > > > >> > > > > www.gridgain.com
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > --
> > > > > > > >> > Sergey Kozlov
> > > > > > > >> > GridGain Systems
> > > > > > > >> > www.gridgain.com
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message