From issues-return-100166-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Tue Oct 8 11:38:03 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 6CA00180658 for ; Tue, 8 Oct 2019 13:38:03 +0200 (CEST) Received: (qmail 73616 invoked by uid 500); 8 Oct 2019 11:38:02 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 73587 invoked by uid 99); 8 Oct 2019 11:38:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Oct 2019 11:38:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 3B4B5E30FA for ; Tue, 8 Oct 2019 11:38:02 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 7F9A37808BD for ; Tue, 8 Oct 2019 11:38:00 +0000 (UTC) Date: Tue, 8 Oct 2019 11:38:00 +0000 (UTC) From: "Maxim Muzafarov (Jira)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (IGNITE-11498) SQL: Rework DML data distribution logic MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/IGNITE-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11498: ------------------------------------- Fix Version/s: (was: 2.8) 2.9 > SQL: Rework DML data distribution logic > --------------------------------------- > > Key: IGNITE-11498 > URL: https://issues.apache.org/jira/browse/IGNITE-11498 > Project: Ignite > Issue Type: Task > Components: sql > Reporter: Vladimir Ozerov > Priority: Major > Fix For: 2.9 > > > Current DML implementation has a number of problems: > 1) We fetch the whole data set to originator's node. There is "skipDmlOnReducer" flag to avoid this in some cases, but it is still in experimental state, and is not enabled by default > 2) Updates are deadlock-prone: we update entries in batches equal to {{SqlFieldsQuery.pageSize}}. So we can deadlock easily with concurrent cache operations > 3) We have very strange re-try logic. It is not clear why it is needed in the first place provided that DML is not transactional and no guarantees are needed. > Proposal: > # Implement proper routing logic: if a request could be executed on data nodes bypassing skipping reducer, do this. Otherwise fetch all data to reducer. This decision should be made in absolutely the same way as for MVCC (see {{GridNearTxQueryEnlistFuture}} as a starting point) > # Distribute updates to primary data node in batches, but apply them one by one, similar to data streamer with {{allowOverwrite=false}}. Do not do any partition state or {{AffinityTopologyVersion}} checks, since DML is not transactional. Return and aggregate update counts back. > # Remove or at least rethink retry logic. Why do we need it in the first place? -- This message was sent by Atlassian Jira (v8.3.4#803005)