Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3B48F200D40 for ; Fri, 3 Nov 2017 12:39:04 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 39F62160BE9; Fri, 3 Nov 2017 11:39:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 826DC160BFB for ; Fri, 3 Nov 2017 12:39:03 +0100 (CET) Received: (qmail 35681 invoked by uid 500); 3 Nov 2017 11:39:02 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 35611 invoked by uid 99); 3 Nov 2017 11:39:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Nov 2017 11:39:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 06513180778 for ; Fri, 3 Nov 2017 11:39:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id pQtZUAvwXUYt for ; Fri, 3 Nov 2017 11:39:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 245AD61140 for ; Fri, 3 Nov 2017 11:39:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6B9E0E095C for ; Fri, 3 Nov 2017 11:39:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 257E023F05 for ; Fri, 3 Nov 2017 11:39:00 +0000 (UTC) Date: Fri, 3 Nov 2017 11:39:00 +0000 (UTC) From: "Ivan Rakov (JIRA)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (IGNITE-5938) Implement WAL logs compaction and compression after checkpoint MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 03 Nov 2017 11:39:04 -0000 [ https://issues.apache.org/jira/browse/IGNITE-5938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov reassigned IGNITE-5938: ---------------------------------- Assignee: Ivan Rakov (was: Dmitriy Govorukhin) > Implement WAL logs compaction and compression after checkpoint > -------------------------------------------------------------- > > Key: IGNITE-5938 > URL: https://issues.apache.org/jira/browse/IGNITE-5938 > Project: Ignite > Issue Type: Improvement > Components: persistence > Affects Versions: 2.1 > Reporter: Alexey Goncharuk > Assignee: Ivan Rakov > Priority: Major > Fix For: 2.4 > > > Currently, we simply move WAL segments to archive when WAL segment is written and delete when checkpoint history becomes too old. > Archived WAL segment contains physical delta records that are no longer needed for rebalancing, so these records may be thrown away. In order to optimize disk space and delta WAL rebalancing, we can do the following: > 1) Clean the WAL segments from the physical records > 2) Compress the cleaned segments (I expect this to be very effective since we write full objects) -- This message was sent by Atlassian JIRA (v6.4.14#64029)