From dev-return-58855-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Sun Dec 1 10:23:03 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2185018061A for ; Sun, 1 Dec 2019 11:23:03 +0100 (CET) Received: (qmail 21310 invoked by uid 500); 1 Dec 2019 10:23:02 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 21273 invoked by uid 99); 1 Dec 2019 10:23:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Dec 2019 10:23:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 8A7FCE2589 for ; Sun, 1 Dec 2019 10:23:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 53E1578047D for ; Sun, 1 Dec 2019 10:23:00 +0000 (UTC) Date: Sun, 1 Dec 2019 10:23:00 +0000 (UTC) From: "chenglei (Jira)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PHOENIX-5494) Batched, mutable Index updates are unnecessarily run one-by-one MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei updated PHOENIX-5494: ------------------------------ Attachment: PHOENIX-5494_v9-master.patch > Batched, mutable Index updates are unnecessarily run one-by-one > --------------------------------------------------------------- > > Key: PHOENIX-5494 > URL: https://issues.apache.org/jira/browse/PHOENIX-5494 > Project: Phoenix > Issue Type: Improvement > Affects Versions: 4.15.0, 5.1.0 > Reporter: Lars Hofhansl > Assignee: chenglei > Priority: Major > Labels: performance > Fix For: 4.15.1, 5.1.1 > > Attachments: 5494-4.x-HBase-1.5.txt, PHOENIX-5494-4.x-HBase-1.4.patch, PHOENIX-5494.master.001.patch, PHOENIX-5494.master.002.patch, PHOENIX-5494.master.003.patch, PHOENIX-5494_v9-4.x-HBase-1.4.patch, PHOENIX-5494_v9-master.patch, Screenshot_20191110_160243.png, Screenshot_20191110_160351.png, Screenshot_20191110_161453.png > > Time Spent: 5h 20m > Remaining Estimate: 0h > > I just noticed that index updates on mutable tables retrieve their deletes (to invalidate the old index entry) one-by-one. > For batches, this can be *the* major time spent during an index update. The cost is mostly incured by the repeated setup (and seeking) of the new region scanner (for each row). > We can instead do a skip scan and get all updates in a single scan per region. > (Logically that is simple, but it will require some refactoring) > I won't be getting to this, but recording it here in case someone feels inclined. -- This message was sent by Atlassian Jira (v8.3.4#803005)