From issues-return-101450-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Mon Nov 11 11:53:03 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 06641180629 for ; Mon, 11 Nov 2019 12:53:02 +0100 (CET) Received: (qmail 56422 invoked by uid 500); 11 Nov 2019 11:53:02 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 56411 invoked by uid 99); 11 Nov 2019 11:53:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Nov 2019 11:53:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B50AFE00D2 for ; Mon, 11 Nov 2019 11:53:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 3D3FD7806B8 for ; Mon, 11 Nov 2019 11:53:00 +0000 (UTC) Date: Mon, 11 Nov 2019 11:53:00 +0000 (UTC) From: "Pavel Pereslegin (Jira)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (IGNITE-11584) Implement batch insertion of new cache entries in FreeList to improve rebalancing MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/IGNITE-11584?page=3Dcom.atlassi= an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16= 971468#comment-16971468 ]=20 Pavel Pereslegin commented on IGNITE-11584: ------------------------------------------- Additional notes on risks and assumptions: # These changes were checked for memory leaks in {{testSimpleRebalancing}}= and {{testComplexRebalancing}} tests, using the following [method|https://= github.com/apache/ignite/commit/472d5aa17de31a0eca780a61f9a655ec20a77e25#di= ff-50ffa3c92f2815b7e31d1dcc22ec6cc1R2329]. # If WAL is enabled during rebalancing, it is possible that after a node f= ailure, the WAL recovery procedure may add unused rows into the page memory= . Expecting partition eviction in such a case. # Rebalancing of "staled" keys can run slower, due to rollbacks. Rollback = occurred when the node already has the key with a newer version - in this c= ase, preloader will remove a "pre-created" data row from the page memory. > Implement batch insertion of new cache entries in FreeList to improve reb= alancing > -------------------------------------------------------------------------= -------- > > Key: IGNITE-11584 > URL: https://issues.apache.org/jira/browse/IGNITE-11584 > Project: Ignite > Issue Type: Sub-task > Affects Versions: 2.7 > Reporter: Pavel Pereslegin > Assignee: Pavel Pereslegin > Priority: Major > Labels: iep-32 > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Main goals: > * Implement batch insert operation into FreeList - insert several data r= ows at once > * Use batch insertion in the preloader > =C2=A0=20 > Implementation notes: > # Preloader cannot lock multiple cache entries at once, because this may= lead to a deadlock with concurrent batch updates. Therefore, it pre-create= s batch of data rows in the page memory, and then sequentially initializes = the cache entries one by one. > # Batch writing of data rows into data pages uses the free list as usual= because other approaches increase memory fragmentation (for example, using= only "reuse" or "most free" buckets). > # Eviction tracker assumes that only data pages with "heads" of fragment= ed data row are tracked, so all other fragments of large data row should be= written on separate data pages (without other data rows which may cause pa= ge tracking). -- This message was sent by Atlassian Jira (v8.3.4#803005)