From dev-return-51163-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Tue Apr 24 21:40:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id D0AB5180671 for ; Tue, 24 Apr 2018 21:40:04 +0200 (CEST) Received: (qmail 92481 invoked by uid 500); 24 Apr 2018 19:40:03 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 92470 invoked by uid 99); 24 Apr 2018 19:40:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Apr 2018 19:40:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 4DF45C015D for ; Tue, 24 Apr 2018 19:40:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id KHxwC_WOUFe1 for ; Tue, 24 Apr 2018 19:40:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 4DF5A5F238 for ; Tue, 24 Apr 2018 19:40:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 730A9E0047 for ; Tue, 24 Apr 2018 19:40:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 63DB1240C6 for ; Tue, 24 Apr 2018 19:40:00 +0000 (UTC) Date: Tue, 24 Apr 2018 19:40:00 +0000 (UTC) From: "James Taylor (JIRA)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450431#comment-16450431 ] James Taylor commented on PHOENIX-3534: --------------------------------------- IMHO, I don't think the propagation of properties from base table to child views will scale. We'd have to update ~2 million rows, trying to get the locks for all of them. Instead, I think we should disallow changes to a base table that need to be propagated to child views until after the splittable system catalog has been rolled out to both server and client (when we'd basically be saying that we won't be rolling back any longer). > Support multi region SYSTEM.CATALOG table > ----------------------------------------- > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug > Reporter: James Taylor > Assignee: Thomas D'Silva > Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region based on the server-side row locks being held for operations that impact a table and all of it's views. For example, adding/removing a column from a base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), when a new table is created we can do a lazy cleanup of any rows that may be left over from a failed DDL call (kudos to [~lhofhansl] for coming up with this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on checking our sequence numbers for each table/view before/after updating. Each table/view row would be individually locked for their change (metadata for a view or table cannot span regions due to our split policy), with the sequence number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)