From dev-return-45928-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Fri May 3 05:10:01 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 03347180671 for ; Fri, 3 May 2019 07:10:00 +0200 (CEST) Received: (qmail 72193 invoked by uid 500); 3 May 2019 05:09:59 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 72179 invoked by uid 99); 3 May 2019 05:09:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 May 2019 05:09:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id B851DC2CB5 for ; Fri, 3 May 2019 05:09:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.157 X-Spam-Level: X-Spam-Status: No, score=0.157 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FROM_EXCESS_BASE64=0.105, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id mETz9X8JP-Vo for ; Fri, 3 May 2019 05:09:55 +0000 (UTC) Received: from mail-ot1-f68.google.com (mail-ot1-f68.google.com [209.85.210.68]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 1E7B65F1CF for ; Fri, 3 May 2019 05:09:54 +0000 (UTC) Received: by mail-ot1-f68.google.com with SMTP id o39so4245499ota.6 for ; Thu, 02 May 2019 22:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=1raIZyECJfN3VgJE6Rjpfr0mMR0pMerlf52RUYaS8l8=; b=YYjpwqbMQoZh5TuIGOc1Fji2Ew3gAIRthG4ORDAV6E5eALnVwOHzOxFwMhi3CfyOrJ aCzPnDyxshcdhEAiCHOCorKKFKbD3rin42isqvPcKO+E8YvHQfg75fplL9CjcVV58KYa tpYz93ChY6BYqdsKikoKHFobF19ImsuCfWQVCrywCX3on51Wsa0FylVqjmLc31XLIVMx 08bdESxsdMaF4baALMzJL4tSVRF2BjLhOzx9DXs7sO+9245TxEC+uGI9VG9cZhnhDFTd 6KU3OY7Se1LvyKTelbdLibZNE5FRexapCi6Tz/i8dS3yGovvrrDmUa5VQKWG5A8d5cq6 QmJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=1raIZyECJfN3VgJE6Rjpfr0mMR0pMerlf52RUYaS8l8=; b=jwGtSnv+CfwqLgZfDtFKiyss/UrI0nu4JGrS3FGASj+QD2rlTQ2THIfz8JXbw1C1vu wLYDF4fFi5ggE7XFrhs3gTEOUQtr+Ys9d8Y+PMuLAY7c5eGuK542YIFPR36M2TaMCQ77 1jG28SgIfP/pZ1Fw/xO21Mz1SCNcEhYt3TY2nkQNxsyC2hmnWjxbfNqk0Ni/37vOHXj+ TMxrw7FUptNRJ74eWCEMqz6m695HPkLq8E0lKgs+IDNS7AG67G4BV8uycbht1djx5tfk o6qux7dRWLJKAuHvhWowSsSRtEAuIp9tzb3CvkDT/nIGjuvwQvE5tSufd7sYCWKJqU2J 3kSQ== X-Gm-Message-State: APjAAAUOW8d3qOqXFWnmzYrHaeg/A5F771Elg+K+zEHIZDmxpEjrG4al rqfQYCczelMzKHt489iYQGEqBfw6ymtPj+WvgFCOah9n X-Google-Smtp-Source: APXvYqxYm9DDovbE0RX8+hN5tbOd/JqZrwCTI9IU3qf9i9yqkxn2vdHcBhFE4PGwgVCNaFwPUfLq5f5vOmyJRZrHtUI= X-Received: by 2002:a05:6830:12d7:: with SMTP id a23mr5572229otq.289.1556860192426; Thu, 02 May 2019 22:09:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?B?0J/QsNCy0LvRg9GF0LjQvSDQmNCy0LDQvQ==?= Date: Fri, 3 May 2019 08:09:42 +0300 Message-ID: Subject: Re: Thin client: transactions support To: dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Alex, I went through IEP [1] and I have a couple of questions: 1. What is going to be used as transaction id? In a described protocol I see an int field for it. Should not it be GridCacheVersion corresponding to IgniteInternalTx#xidVersion? 2. OP_TX_END message assumes an empty response, but I think that errors during tx finish are possible and should be returned in a response. 3. In IEP it is stated that async processing of lock operations should be introduced on a client side to enable concurrent operations from different client threads. Do you have an idea how to achieve it? Also, a bit about a suspend/resume trait. I tried to think about it leaving away an existing transactions implementation in Ignite. As I understood we are going to resume a tx before each cache operation in the tx and resume the tx after the operation. All this to make an executing thread available for other operations (e.g. in other txs). From the first glance it seems like an inversed logic. A straightforward way is to execute a cache operation within a particular transaction defined as an explicit tx id argument (e.g. cache.put(key, value, txid)). Can we do so? And leaving for now thin client API. I cannot say that one proposed in IEP is good or bad. I can only say that it ressembles current thick client API. And perhaps it should not. I think that we should consider similar APIs provided by other vendors and keep in mind that we have a bunch of client implementations for different languages. I suppose that we can return to it a little bit later. And I hope that we will do it. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3= A+transactions+support =D0=B2=D1=82, 30 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 13:24, Alex Plehan= ov : > > Hello, Igniters! > > I've update IEP [1] and implement PoC according to new approach (multiple > concurrent transactions per connection). > But to move forward another feature need to be implemented: suspend/resum= e > for pessimistic transactions (IGNITE-5714 [2]). Implementation of > suspend/resume is ready now and ticket in 'Patch available' status. Can a= ny > transactions expert help with review of IGNITE-5714? > > [1]: > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+= transactions+support > [2]: https://issues.apache.org/jira/browse/IGNITE-5714 > > =D1=87=D1=82, 4 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 11:32, Alex Pleha= nov : > > > Vladimir, > > > > Ok, then I will rewrite IEP in the near future. > > > > =D1=87=D1=82, 4 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 11:14, Vladimir= Ozerov : > > > >> Hi Alex, > >> > >> I think we should be able to handle many transactions through a single > >> connection. This will make our protocol and client implementations muc= h > >> more efficient, and simplicity from developer's perspective is not our > >> goal. Consider normal nodes. We have server nodes and client nodes. Yo= u > >> may > >> span whatever number of transactions you need, but all of them are > >> coordinated through a single connection. The same should be applicable= to > >> thin clients. Protocol is already designed to handle this, as we pass > >> unique operation ID in order to distinguish one operation from another= . It > >> is true, though, that we will have to introduce a kind of "session" > >> concept, and pass additional identifier along with cache operations, b= ut > >> this doesn't sound like a problem to me. > >> > >> And provided that currently server-side transactions are bound to thre= ads > >> artificially, I would say that the first step in implementation of > >> transactions on thin clients should be decoupling server-side transact= ions > >> from threads. Without this we will have very inefficient implementatio= n, > >> when every new client transaction have to spawn a new thread. This is = slow > >> and introduces high memory pressure on a cluster node. We already work > >> this > >> way for MVCC transactions which are spawned from JDBC driver, and beli= eve > >> me, we do not want to replicated this bad practice to other clients :-= ) > >> > >> Vladimir. > >> > >> On Thu, Apr 4, 2019 at 10:08 AM Alex Plehanov > >> wrote: > >> > >> > Guys, so, do we need multiple concurrent transactions per connection= ? > >> > > >> > There are pros and cons for each approach. Difference between > >> approaches: > >> > > >> > One transaction at a time per connection: > >> > - This approach is used in RDBMS world and users got used to it > >> > - To use transactions concurrently users need to use different > >> connections > >> > and get these connections via something like a connection pool > >> > - Easy to implement (in fact, PoC is already done) > >> > > >> > Multiple concurrent transactions per connection: > >> > - At least for java thin client, we can implement transaction per > >> thread > >> > approach as implemented now for the thick client (perhaps other thin > >> > clients can implement the same abstraction) > >> > - There is also protocol change for all cache operations needed (to > >> bind > >> > cache operation to the transaction) > >> > - Significant changes to all implemented clients are needed > >> > - Implementation on the server side is more complex > >> > > >> > What do you think? > >> > > >> > > >> > =D0=B2=D1=82, 2 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 16:29, Alex = Plehanov : > >> > > >> > > Ilya, > >> > > > >> > > > We should be able to multiplex several transactions using a sing= le > >> > > Client connection. > >> > > In this case, we should significantly change cache operations synt= ax > >> (for > >> > > each implemented client), to bind each operation to the transactio= n. > >> > > > >> > > > I want to also ask if "Number of entries participating in > >> transaction > >> > > (may be approximate). 0 - default value." is needed. > >> > > I've tried to minimize API changes between thick and thin client t= o > >> > > simplify move from one to another. It's the only reason. But I agr= ee > >> with > >> > > you, the parameter is not very useful. > >> > > > >> > > > >> > > =D0=B2=D1=82, 2 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 14:48, Ily= a Kasnacheev < > >> ilya.kasnacheev@gmail.com>: > >> > > > >> > >> Hello! > >> > >> > >> > >> Pavel, I agree with you thorougly. We should be able to multiplex > >> > several > >> > >> transactions using a single Client connection. This means adding > >> > >> Transaction id parameter to every affected cache operation / SQL > >> > statement > >> > >> (if applicable) to make sure we do cache operations on relevant > >> > >> transaction. > >> > >> > >> > >> This is how other things work in Ignite, such as communication. W= e do > >> > not > >> > >> open dozens of connections, we multiplex operations asynchronousl= y > >> > through > >> > >> a single connection. > >> > >> > >> > >> I think that trying to pool Ignite connections will be highly > >> > >> inconvenient, > >> > >> since there is no existing infrastructure for such pooling (like > >> there > >> > >> exists for JDBC). > >> > >> > >> > >> I want to also ask if "Number of entries participating in transac= tion > >> > (may > >> > >> be approximate). 0 - default value." is needed. Does it actually = do > >> > >> anything in our tx protocol? Users of existing APIs are already > >> confused > >> > >> by > >> > >> this parameter, if we could get rid of it in thin client protocol= it > >> > would > >> > >> be nice clean-up. > >> > >> > >> > >> Regards, > >> > >> -- > >> > >> Ilya Kasnacheev > >> > >> > >> > >> > >> > >> =D0=B2=D1=82, 2 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 09:55, Pa= vel Tupitsyn : > >> > >> > >> > >> > Alex, > >> > >> > > >> > >> > > now we can only support one active transaction per connection > >> > >> > > >> > >> > I totally understand server-side and protocol limitations that = are > >> > >> causing > >> > >> > this. > >> > >> > But I have no idea how to support this in .NET Thin Client, for > >> > example. > >> > >> > > >> > >> > It is thread-safe and can handle multiple async operations in > >> > parallel. > >> > >> > But with TX support we have to somehow switch to single-threade= d > >> mode > >> > to > >> > >> > avoid unexpected effects. > >> > >> > > >> > >> > Any ideas? > >> > >> > > >> > >> > > >> > >> > On Mon, Apr 1, 2019 at 6:38 PM Alex Plehanov < > >> plehanov.alex@gmail.com > >> > > > >> > >> > wrote: > >> > >> > > >> > >> > > Dmitriy, thank you! > >> > >> > > > >> > >> > > Guys, I've created the IEP [1] on wiki, please have a look. > >> > >> > > > >> > >> > > [1] > >> > >> > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%= 3A+transactions+support > >> > >> > > > >> > >> > > > >> > >> > > =D1=87=D1=82, 28 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0=B2 14:3= 3, Dmitriy Pavlov >> >: > >> > >> > > > >> > >> > > > Hi, > >> > >> > > > > >> > >> > > > I've added permissions to account plehanov.alex > >> > >> > > > > >> > >> > > > Recently Infra integrated Apache LDAP with confluence, so i= t is > >> > >> > possible > >> > >> > > to > >> > >> > > > login using Apache credentials. Probably we can ask infra i= f > >> extra > >> > >> > > > permissions to edit pages should be added for committers. > >> > >> > > > > >> > >> > > > Sincerely, > >> > >> > > > Dmitriy Pavlov > >> > >> > > > > >> > >> > > > =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0=B2 13= :37, Alex Plehanov < > >> > plehanov.alex@gmail.com > >> > >> >: > >> > >> > > > > >> > >> > > > > Vladimir, > >> > >> > > > > > >> > >> > > > > About current tx: ok, then we don't need tx() method in t= he > >> > >> interface > >> > >> > > at > >> > >> > > > > all (the same cached transaction info user can store by > >> > himself). > >> > >> > > > > > >> > >> > > > > About decoupling transactions from threads on the server > >> side: > >> > for > >> > >> > now, > >> > >> > > > we > >> > >> > > > > can start with thread-per-connection approach (we only ca= n > >> > support > >> > >> > one > >> > >> > > > > active transaction per connection, see below, so we need = one > >> > >> > additional > >> > >> > > > > dedicated thread for each connection with active > >> transaction), > >> > and > >> > >> > > later > >> > >> > > > > change server-side internals to process client transactio= ns > >> in > >> > any > >> > >> > > server > >> > >> > > > > thread (not dedicated to this connection). This change wi= ll > >> not > >> > >> > affect > >> > >> > > > the > >> > >> > > > > thin client protocol, it only affects the server side. > >> > >> > > > > In any case, we can't support concurrent transactions per > >> > >> connection > >> > >> > on > >> > >> > > > > the client side without fundamental changes to the curren= t > >> > >> protocol > >> > >> > > > (cache > >> > >> > > > > operation doesn't bound to transaction or thread and the > >> server > >> > >> > doesn't > >> > >> > > > > know which thread on the client side do this cache > >> operation). > >> > In > >> > >> my > >> > >> > > > > opinion, if a user wants to use concurrent transactions, = he > >> must > >> > >> use > >> > >> > > > > different connections from a connection pool. > >> > >> > > > > > >> > >> > > > > About semantics of suspend/resume on the client-side: it'= s > >> > >> absolutely > >> > >> > > > > different than server-side semantics (we don't need to do > >> > >> > > suspend/resume > >> > >> > > > to > >> > >> > > > > pass transaction between threads on the client-side), but > >> can't > >> > be > >> > >> > > > > implemented efficiently without implemented suspend/resum= e on > >> > >> > > > server-side. > >> > >> > > > > > >> > >> > > > > Can anyone give me permissions to create IEP on Apache wi= ki? > >> > >> > > > > > >> > >> > > > > =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0=B2 = 11:59, Vladimir Ozerov < > >> > >> vozerov@gridgain.com>: > >> > >> > > > > > >> > >> > > > > > Hi Alex, > >> > >> > > > > > > >> > >> > > > > > My comments was only about the protocol. Getting curren= t > >> info > >> > >> about > >> > >> > > > > > transaction should be handled by the client itself. It = is > >> not > >> > >> > > protocl's > >> > >> > > > > > concern. Same about other APIs and behavior in case ano= ther > >> > >> > > transaction > >> > >> > > > > is > >> > >> > > > > > attempted from the same thread. > >> > >> > > > > > > >> > >> > > > > > Putting protocol aside, transaction support is complica= ted > >> > >> matter. > >> > >> > I > >> > >> > > > > would > >> > >> > > > > > propose to route through IEP and wide community > >> discussion. We > >> > >> need > >> > >> > > to > >> > >> > > > > > review API and semantics very carefully, taking > >> SUSPEND/RESUME > >> > >> in > >> > >> > > > count. > >> > >> > > > > > Also I do not see how we support client transactions > >> > efficiently > >> > >> > > > without > >> > >> > > > > > decoupling transactions from threads on the server side > >> first. > >> > >> > > Because > >> > >> > > > > > without it you will need a dedicated server thread for > >> every > >> > >> > client's > >> > >> > > > > > transaction which is slow and may even crash the server= . > >> > >> > > > > > > >> > >> > > > > > Vladimir. > >> > >> > > > > > > >> > >> > > > > > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov < > >> > >> > > > plehanov.alex@gmail.com> > >> > >> > > > > > wrote: > >> > >> > > > > > > >> > >> > > > > > > Vladimir, what if we want to get current transaction = info > >> > >> (tx() > >> > >> > > > > method)? > >> > >> > > > > > > > >> > >> > > > > > > Does close() method mapped to TX_END(rollback)? > >> > >> > > > > > > > >> > >> > > > > > > For example, this code: > >> > >> > > > > > > > >> > >> > > > > > > try(tx =3D txStart()) { > >> > >> > > > > > > tx.commit(); > >> > >> > > > > > > } > >> > >> > > > > > > > >> > >> > > > > > > Will produce: > >> > >> > > > > > > TX_START > >> > >> > > > > > > TX_END(commit) > >> > >> > > > > > > TX_END(rollback) > >> > >> > > > > > > > >> > >> > > > > > > Am I understand you right? > >> > >> > > > > > > > >> > >> > > > > > > About xid. There is yet another proposal. Use some un= ique > >> > per > >> > >> > > > > connection > >> > >> > > > > > id > >> > >> > > > > > > (integer, simple counter) for identifying the > >> transaction on > >> > >> > > > > > > commit/rollback message. The client gets this id from= the > >> > >> server > >> > >> > > with > >> > >> > > > > > > transaction info and sends it back to the server when > >> trying > >> > >> to > >> > >> > > > > > > commit/rollback transaction. This id is not shown to > >> users. > >> > >> But > >> > >> > > also > >> > >> > > > we > >> > >> > > > > > can > >> > >> > > > > > > pass from server to client real transaction id (xid) = with > >> > >> > > transaction > >> > >> > > > > > info > >> > >> > > > > > > for diagnostic purposes. > >> > >> > > > > > > > >> > >> > > > > > > And one more question: what should we do if the clien= t > >> > starts > >> > >> a > >> > >> > new > >> > >> > > > > > > transaction without ending the old one? Should we end= the > >> > old > >> > >> > > > > transaction > >> > >> > > > > > > implicitly (rollback) or throw an exception to the > >> client? > >> > In > >> > >> my > >> > >> > > > > opinion, > >> > >> > > > > > > the first option is better. For example, if we got a > >> > >> previously > >> > >> > > used > >> > >> > > > > > > connection from the connection pool, we should not wo= rry > >> > about > >> > >> > any > >> > >> > > > > > > uncompleted transaction started by the previous user = of > >> this > >> > >> > > > > connection. > >> > >> > > > > > > > >> > >> > > > > > > =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0= =B2 11:02, Vladimir Ozerov < > >> > >> > vozerov@gridgain.com > >> > >> > > >: > >> > >> > > > > > > > >> > >> > > > > > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not supp= ort > >> > them > >> > >> > yet, > >> > >> > > > and > >> > >> > > > > > > adding > >> > >> > > > > > > > them in future should not conflict with simple > >> START/END > >> > >> > > > > > infrastructure. > >> > >> > > > > > > > > >> > >> > > > > > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov < > >> > >> > > > > vozerov@gridgain.com > >> > >> > > > > > > > >> > >> > > > > > > > wrote: > >> > >> > > > > > > > > >> > >> > > > > > > > > Hi Alex, > >> > >> > > > > > > > > > >> > >> > > > > > > > > I am not sure we need 5 commands. Wouldn't it be > >> enough > >> > to > >> > >> > have > >> > >> > > > > only > >> > >> > > > > > > two? > >> > >> > > > > > > > > > >> > >> > > > > > > > > START - accepts optional parameters, returns > >> transaction > >> > >> info > >> > >> > > > > > > > > END - provides commit flag, returns void > >> > >> > > > > > > > > > >> > >> > > > > > > > > Vladimir. > >> > >> > > > > > > > > > >> > >> > > > > > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov < > >> > >> > > > > > plehanov.alex@gmail.com > >> > >> > > > > > > > > >> > >> > > > > > > > > wrote: > >> > >> > > > > > > > > > >> > >> > > > > > > > >> Sergey, yes, the close is something like silent > >> > rollback. > >> > >> > But > >> > >> > > we > >> > >> > > > > can > >> > >> > > > > > > > >> also implement this on the client side, just usi= ng > >> > >> rollback > >> > >> > > and > >> > >> > > > > > > ignoring > >> > >> > > > > > > > >> errors in the response. > >> > >> > > > > > > > >> > >> > >> > > > > > > > >> =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3= . =D0=B2 00:04, Sergey Kozlov < > >> > >> > > > skozlov@gridgain.com > >> > >> > > > > >: > >> > >> > > > > > > > >> > >> > >> > > > > > > > >> > Nikolay > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > Am I correctly understand you points: > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > - close: rollback > >> > >> > > > > > > > >> > - commit, close: do nothing > >> > >> > > > > > > > >> > - rollback, close: do what? (I suppose noth= ing) > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > Also you assume that after commit/rollback we = may > >> > need > >> > >> to > >> > >> > > free > >> > >> > > > > > some > >> > >> > > > > > > > >> > resources on server node(s)or just do on clien= t > >> > started > >> > >> > TX? > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex Plehanov= < > >> > >> > > > > > > > plehanov.alex@gmail.com > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > wrote: > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > Sergey, we have the close() method in the th= ick > >> > >> client, > >> > >> > > it's > >> > >> > > > > > > > behavior > >> > >> > > > > > > > >> is > >> > >> > > > > > > > >> > > slightly different than rollback() method (i= t > >> > should > >> > >> > > > rollback > >> > >> > > > > if > >> > >> > > > > > > the > >> > >> > > > > > > > >> > > transaction is not committed and do nothing = if > >> the > >> > >> > > > transaction > >> > >> > > > > > is > >> > >> > > > > > > > >> already > >> > >> > > > > > > > >> > > committed). I think we should support > >> > >> try-with-resource > >> > >> > > > > > semantics > >> > >> > > > > > > in > >> > >> > > > > > > > >> the > >> > >> > > > > > > > >> > > thin client and OP_TX_CLOSE will be useful h= ere. > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > Nikolay, suspend/resume didn't work yet for > >> > >> pessimistic > >> > >> > > > > > > > transactions. > >> > >> > > > > > > > >> > Also, > >> > >> > > > > > > > >> > > the main goal of suspend/resume operations i= s to > >> > >> support > >> > >> > > > > > > transaction > >> > >> > > > > > > > >> > > passing between threads. In the thin client,= the > >> > >> > > transaction > >> > >> > > > > is > >> > >> > > > > > > > bound > >> > >> > > > > > > > >> to > >> > >> > > > > > > > >> > > the client connection, not client thread. I > >> think > >> > >> > passing > >> > >> > > a > >> > >> > > > > > > > >> transaction > >> > >> > > > > > > > >> > > between different client connections is not = a > >> very > >> > >> > useful > >> > >> > > > > case. > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > =D0=B2=D1=82, 26 =D0=BC=D0=B0=D1=80. 2019 = =D0=B3. =D0=B2 22:17, Nikolay Izhikov < > >> > >> > > > > > nizhikov@apache.org > >> > >> > > > > > > >: > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > > Hello, Alex. > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > We also have suspend and resume operations= . > >> > >> > > > > > > > >> > > > I think we should support them > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > =D0=B2=D1=82, 26 =D0=BC=D0=B0=D1=80=D1=82= =D0=B0 2019 =D0=B3., 22:07 Sergey Kozlov < > >> > >> > > > > > skozlov@gridgain.com > >> > >> > > > > > > >: > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > > Hi > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > Looks like I missed something but why we > >> need > >> > >> > > > OP_TX_CLOSE > >> > >> > > > > > > > >> operation? > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > Also I suggest to reserve a code for > >> SAVEPOINT > >> > >> > > operation > >> > >> > > > > > which > >> > >> > > > > > > > >> very > >> > >> > > > > > > > >> > > > useful > >> > >> > > > > > > > >> > > > > to understand where transaction has been > >> rolled > >> > >> back > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > On Tue, Mar 26, 2019 at 6:07 PM Alex > >> Plehanov < > >> > >> > > > > > > > >> > plehanov.alex@gmail.com > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > > wrote: > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > > Hello Igniters! > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > I want to pick up the ticket IGNITE-73= 69 > >> and > >> > >> add > >> > >> > > > > > > transactions > >> > >> > > > > > > > >> > support > >> > >> > > > > > > > >> > > > to > >> > >> > > > > > > > >> > > > > > our thin client implementation. > >> > >> > > > > > > > >> > > > > > I've looked at our current implementat= ion > >> and > >> > >> have > >> > >> > > > some > >> > >> > > > > > > > >> proposals > >> > >> > > > > > > > >> > to > >> > >> > > > > > > > >> > > > > > support transactions: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > Add new operations to thin client > >> protocol: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > OP_TX_GET, 4000, Get current > >> transaction > >> > >> for > >> > >> > > > client > >> > >> > > > > > > > >> connection > >> > >> > > > > > > > >> > > > > > OP_TX_START, 4001, Start a new > >> > transaction > >> > >> > > > > > > > >> > > > > > OP_TX_COMMIT, 4002, Commit transac= tion > >> > >> > > > > > > > >> > > > > > OP_TX_ROLLBACK, 4003, Rollback > >> > transaction > >> > >> > > > > > > > >> > > > > > OP_TX_CLOSE, 4004, Close transacti= on > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > From the client side (java) new interf= aces > >> > >> will be > >> > >> > > > > added: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > public interface ClientTransactions { > >> > >> > > > > > > > >> > > > > > public ClientTransaction txStart()= ; > >> > >> > > > > > > > >> > > > > > public ClientTransaction > >> > >> > > > > > txStart(TransactionConcurrency > >> > >> > > > > > > > >> > > > concurrency, > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation); > >> > >> > > > > > > > >> > > > > > public ClientTransaction > >> > >> > > > > > txStart(TransactionConcurrency > >> > >> > > > > > > > >> > > > concurrency, > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation, long > >> timeout, > >> > >> int > >> > >> > > > > txSize); > >> > >> > > > > > > > >> > > > > > public ClientTransaction tx(); // = Get > >> > >> current > >> > >> > > > > > connection > >> > >> > > > > > > > >> > > > transaction > >> > >> > > > > > > > >> > > > > > public ClientTransactions > >> > withLabel(String > >> > >> > lb); > >> > >> > > > > > > > >> > > > > > } > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > public interface ClientTransaction ext= ends > >> > >> > > > > AutoCloseable { > >> > >> > > > > > > > >> > > > > > public IgniteUuid xid(); // Do we = need > >> > it? > >> > >> > > > > > > > >> > > > > > public TransactionIsolation > >> isolation(); > >> > >> > > > > > > > >> > > > > > public TransactionConcurrency > >> > >> concurrency(); > >> > >> > > > > > > > >> > > > > > public long timeout(); > >> > >> > > > > > > > >> > > > > > public String label(); > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > public void commit(); > >> > >> > > > > > > > >> > > > > > public void rollback(); > >> > >> > > > > > > > >> > > > > > public void close(); > >> > >> > > > > > > > >> > > > > > } > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > From the server side, I think as a fir= st > >> step > >> > >> > (while > >> > >> > > > > > > > >> transactions > >> > >> > > > > > > > >> > > > > > suspend/resume is not fully implemente= d) > >> we > >> > can > >> > >> > use > >> > >> > > > the > >> > >> > > > > > same > >> > >> > > > > > > > >> > approach > >> > >> > > > > > > > >> > > > as > >> > >> > > > > > > > >> > > > > > for JDBC: add a new worker to each > >> > >> > > > ClientRequestHandler > >> > >> > > > > > and > >> > >> > > > > > > > >> process > >> > >> > > > > > > > >> > > > > > requests by this worker if the > >> transaction is > >> > >> > > started > >> > >> > > > > > > > >> explicitly. > >> > >> > > > > > > > >> > > > > > ClientRequestHandler is bound to clien= t > >> > >> > connection, > >> > >> > > so > >> > >> > > > > > there > >> > >> > > > > > > > >> will > >> > >> > > > > > > > >> > be > >> > >> > > > > > > > >> > > > 1:1 > >> > >> > > > > > > > >> > > > > > relation between client connection and > >> > thread, > >> > >> > which > >> > >> > > > > > process > >> > >> > > > > > > > >> > > operations > >> > >> > > > > > > > >> > > > > in > >> > >> > > > > > > > >> > > > > > a transaction. > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > Also, there is a couple of issues I wa= nt > >> to > >> > >> > discuss: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > We have overloaded method txStart with= a > >> > >> different > >> > >> > > set > >> > >> > > > > of > >> > >> > > > > > > > >> > arguments. > >> > >> > > > > > > > >> > > > Some > >> > >> > > > > > > > >> > > > > > of the arguments may be missing. To pa= ss > >> > >> arguments > >> > >> > > > with > >> > >> > > > > > > > >> OP_TX_START > >> > >> > > > > > > > >> > > > > > operation we have the next options: > >> > >> > > > > > > > >> > > > > > * Serialize full set of arguments and= use > >> > some > >> > >> > > value > >> > >> > > > > for > >> > >> > > > > > > > >> missing > >> > >> > > > > > > > >> > > > > > arguments. For example -1 for int/long > >> types > >> > >> and > >> > >> > > null > >> > >> > > > > for > >> > >> > > > > > > > string > >> > >> > > > > > > > >> > > type. > >> > >> > > > > > > > >> > > > We > >> > >> > > > > > > > >> > > > > > can't use 0 for int/long types since 0 > >> it's a > >> > >> > valid > >> > >> > > > > value > >> > >> > > > > > > for > >> > >> > > > > > > > >> > > > > concurrency, > >> > >> > > > > > > > >> > > > > > isolation and timeout arguments. > >> > >> > > > > > > > >> > > > > > * Serialize arguments as a collection= of > >> > >> > > > property-value > >> > >> > > > > > > pairs > >> > >> > > > > > > > >> > (like > >> > >> > > > > > > > >> > > > it's > >> > >> > > > > > > > >> > > > > > implemented now for CacheConfiguration= ). > >> In > >> > >> this > >> > >> > > case > >> > >> > > > > only > >> > >> > > > > > > > >> > explicitly > >> > >> > > > > > > > >> > > > > > provided arguments will be serialized. > >> > >> > > > > > > > >> > > > > > Which way is better? The simplest > >> solution is > >> > >> to > >> > >> > use > >> > >> > > > the > >> > >> > > > > > > first > >> > >> > > > > > > > >> > option > >> > >> > > > > > > > >> > > > > and I > >> > >> > > > > > > > >> > > > > > want to use it if there were no > >> objections. > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > Do we need transaction id (xid) on the > >> client > >> > >> > side? > >> > >> > > > > > > > >> > > > > > If yes, we can pass xid along with > >> > >> OP_TX_COMMIT, > >> > >> > > > > > > > OP_TX_ROLLBACK, > >> > >> > > > > > > > >> > > > > > OP_TX_CLOSE operations back to the ser= ver > >> and > >> > >> do > >> > >> > > > > > additional > >> > >> > > > > > > > >> check > >> > >> > > > > > > > >> > on > >> > >> > > > > > > > >> > > > the > >> > >> > > > > > > > >> > > > > > server side (current transaction id fo= r > >> > >> connection > >> > >> > > =3D=3D > >> > >> > > > > > > > >> transaction > >> > >> > > > > > > > >> > id > >> > >> > > > > > > > >> > > > > passed > >> > >> > > > > > > > >> > > > > > from client side). This, perhaps, will > >> > protect > >> > >> > > clients > >> > >> > > > > > > against > >> > >> > > > > > > > >> some > >> > >> > > > > > > > >> > > > > errors > >> > >> > > > > > > > >> > > > > > (for example when client try to commit > >> > outdated > >> > >> > > > > > > transaction). > >> > >> > > > > > > > >> But > >> > >> > > > > > > > >> > > > > > currently, we don't have data type > >> IgniteUuid > >> > >> in > >> > >> > > thin > >> > >> > > > > > client > >> > >> > > > > > > > >> > > protocol. > >> > >> > > > > > > > >> > > > Do > >> > >> > > > > > > > >> > > > > > we need to add it too? > >> > >> > > > > > > > >> > > > > > Also, we can pass xid as a string just= to > >> > >> inform > >> > >> > the > >> > >> > > > > > client > >> > >> > > > > > > > and > >> > >> > > > > > > > >> do > >> > >> > > > > > > > >> > > not > >> > >> > > > > > > > >> > > > > pass > >> > >> > > > > > > > >> > > > > > it back to the server with commit/roll= back > >> > >> > > operation. > >> > >> > > > > > > > >> > > > > > Or not to pass xid at all (.NET thick > >> client > >> > >> works > >> > >> > > > this > >> > >> > > > > > way > >> > >> > > > > > > as > >> > >> > > > > > > > >> far > >> > >> > > > > > > > >> > > as I > >> > >> > > > > > > > >> > > > > > know). > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > What do you think? > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > =D1=81=D1=80, 7 =D0=BC=D0=B0=D1=80. 20= 18 =D0=B3. =D0=B2 16:22, Vladimir > >> Ozerov < > >> > >> > > > > > > > >> vozerov@gridgain.com > >> > >> > > > > > > > >> > >: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > > We already have transactions support= in > >> > JDBC > >> > >> > > driver > >> > >> > > > in > >> > >> > > > > > TX > >> > >> > > > > > > > SQL > >> > >> > > > > > > > >> > > branch > >> > >> > > > > > > > >> > > > > > > (ignite-4191). Currently it is > >> implemented > >> > >> > through > >> > >> > > > > > > separate > >> > >> > > > > > > > >> > thread, > >> > >> > > > > > > > >> > > > > which > >> > >> > > > > > > > >> > > > > > > is not that efficient. Ideally we ne= ed > >> to > >> > >> finish > >> > >> > > > > > > decoupling > >> > >> > > > > > > > >> > > > > transactions > >> > >> > > > > > > > >> > > > > > > from threads. But alternatively we c= an > >> > change > >> > >> > the > >> > >> > > > > logic > >> > >> > > > > > on > >> > >> > > > > > > > >> how we > >> > >> > > > > > > > >> > > > > assign > >> > >> > > > > > > > >> > > > > > > thread ID to specific transaction an= d > >> > >> > > "impersonate" > >> > >> > > > > thin > >> > >> > > > > > > > >> client > >> > >> > > > > > > > >> > > > worker > >> > >> > > > > > > > >> > > > > > > threads when serving requests from > >> multiple > >> > >> > users. > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > On Tue, Mar 6, 2018 at 10:01 PM, Den= is > >> > Magda > >> > >> < > >> > >> > > > > > > > >> dmagda@apache.org> > >> > >> > > > > > > > >> > > > > wrote: > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > Here is an original discussion wit= h a > >> > >> > reference > >> > >> > > to > >> > >> > > > > the > >> > >> > > > > > > > JIRA > >> > >> > > > > > > > >> > > ticket: > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > http://apache-ignite-developers.2346864.n4.nabble > >> > >> > > > . > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > com/Re-Transaction-operations-using-the-Ignite-Thin-Client- > >> > >> > > > > > > > >> > > > > > > > Protocol-td25914.html > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > -- > >> > >> > > > > > > > >> > > > > > > > Denis > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > On Tue, Mar 6, 2018 at 9:18 AM, > >> Dmitriy > >> > >> > > Setrakyan > >> > >> > > > < > >> > >> > > > > > > > >> > > > > > dsetrakyan@apache.org > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > wrote: > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > Hi Dmitriy. I don't think we hav= e a > >> > >> design > >> > >> > > > > proposal > >> > >> > > > > > > for > >> > >> > > > > > > > >> > > > transaction > >> > >> > > > > > > > >> > > > > > > > support > >> > >> > > > > > > > >> > > > > > > > > in thin clients. Do you mind tak= ing > >> > this > >> > >> > > > > initiative > >> > >> > > > > > > and > >> > >> > > > > > > > >> > > creating > >> > >> > > > > > > > >> > > > an > >> > >> > > > > > > > >> > > > > > IEP > >> > >> > > > > > > > >> > > > > > > > on > >> > >> > > > > > > > >> > > > > > > > > Wiki? > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > D. > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > On Tue, Mar 6, 2018 at 8:46 AM, > >> Dmitriy > >> > >> > > > > Govorukhin < > >> > >> > > > > > > > >> > > > > > > > > dmitriy.govorukhin@gmail.com> > >> wrote: > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > Hi, Igniters. > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > I've seen a lot of discussions > >> about > >> > >> thin > >> > >> > > > client > >> > >> > > > > > and > >> > >> > > > > > > > >> binary > >> > >> > > > > > > > >> > > > > > protocol, > >> > >> > > > > > > > >> > > > > > > > > but I > >> > >> > > > > > > > >> > > > > > > > > > did not hear anything about > >> > >> transactions > >> > >> > > > > support. > >> > >> > > > > > Do > >> > >> > > > > > > > we > >> > >> > > > > > > > >> > have > >> > >> > > > > > > > >> > > > some > >> > >> > > > > > > > >> > > > > > > draft > >> > >> > > > > > > > >> > > > > > > > > for > >> > >> > > > > > > > >> > > > > > > > > > this purpose? > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > As I understand we have severa= l > >> > >> problems: > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > - thread and transaction ha= ve > >> hard > >> > >> > > related > >> > >> > > > > (we > >> > >> > > > > > > use > >> > >> > > > > > > > >> > > > > thread-local > >> > >> > > > > > > > >> > > > > > > > > variable > >> > >> > > > > > > > >> > > > > > > > > > and thread name) > >> > >> > > > > > > > >> > > > > > > > > > - we can process only one > >> > >> transaction > >> > >> > at > >> > >> > > > the > >> > >> > > > > > same > >> > >> > > > > > > > >> time > >> > >> > > > > > > > >> > in > >> > >> > > > > > > > >> > > > one > >> > >> > > > > > > > >> > > > > > > thread > >> > >> > > > > > > > >> > > > > > > > > (it > >> > >> > > > > > > > >> > > > > > > > > > mean we need hold thread pe= r > >> > >> client. If > >> > >> > > > > connect > >> > >> > > > > > > 100 > >> > >> > > > > > > > >> thin > >> > >> > > > > > > > >> > > > > clients > >> > >> > > > > > > > >> > > > > > > to > >> > >> > > > > > > > >> > > > > > > > 1 > >> > >> > > > > > > > >> > > > > > > > > > server node, then need to h= old > >> 100 > >> > >> > thread > >> > >> > > > on > >> > >> > > > > > the > >> > >> > > > > > > > >> server > >> > >> > > > > > > > >> > > > side) > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > Let's discuss how we can imple= ment > >> > >> > > > transactions > >> > >> > > > > > for > >> > >> > > > > > > > the > >> > >> > > > > > > > >> > thin > >> > >> > > > > > > > >> > > > > > client. > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > -- > >> > >> > > > > > > > >> > > > > Sergey Kozlov > >> > >> > > > > > > > >> > > > > GridGain Systems > >> > >> > > > > > > > >> > > > > www.gridgain.com > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > -- > >> > >> > > > > > > > >> > Sergey Kozlov > >> > >> > > > > > > > >> > GridGain Systems > >> > >> > > > > > > > >> > www.gridgain.com > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > >> > >> > > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > > >> > > >> > > --=20 Best regards, Ivan Pavlukhin