From dev-return-52121-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Thu May 31 00:02:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 00B91180676 for ; Thu, 31 May 2018 00:02:03 +0200 (CEST) Received: (qmail 71541 invoked by uid 500); 30 May 2018 22:02:03 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 71523 invoked by uid 99); 30 May 2018 22:02:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 May 2018 22:02:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 89B14180314 for ; Wed, 30 May 2018 22:02:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id snUcBfqDH2IO for ; Wed, 30 May 2018 22:02:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 131ED5F357 for ; Wed, 30 May 2018 22:02:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 7D76DE02DB for ; Wed, 30 May 2018 22:02:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 36EB721095 for ; Wed, 30 May 2018 22:02:00 +0000 (UTC) Date: Wed, 30 May 2018 22:02:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495751#comment-16495751 ] ASF GitHub Bot commented on PHOENIX-3534: ----------------------------------------- Github user JamesRTaylor commented on a diff in the pull request: https://github.com/apache/phoenix/pull/303#discussion_r191936918 --- Diff: phoenix-core/src/test/java/org/apache/phoenix/coprocessor/MetaDataEndpointImplTest.java --- @@ -0,0 +1,299 @@ +package org.apache.phoenix.coprocessor; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.fail; + +import java.io.IOException; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.SQLException; +import java.util.Arrays; +import java.util.List; +import java.util.Map; + +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.client.HTable; +import org.apache.phoenix.end2end.ParallelStatsDisabledIT; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.TableNotFoundException; +import org.apache.phoenix.util.PhoenixRuntime; +import org.junit.Test; + +import com.google.common.base.Joiner; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Lists; +import com.google.common.collect.Maps; + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * http://www.apache.org/licenses/LICENSE-2.0 + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +public class MetaDataEndpointImplTest extends ParallelStatsDisabledIT { --- End diff -- One test that would be useful (and maybe you already have it?) is to create a table, create a view (maybe on a different RS). Then drop the table and recreate it and the view with the same name but different columns. Make sure that the lazy cleanup code cleaned up the left over state correctly. > Support multi region SYSTEM.CATALOG table > ----------------------------------------- > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug > Reporter: James Taylor > Assignee: Thomas D'Silva > Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region based on the server-side row locks being held for operations that impact a table and all of it's views. For example, adding/removing a column from a base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), when a new table is created we can do a lazy cleanup of any rows that may be left over from a failed DDL call (kudos to [~lhofhansl] for coming up with this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on checking our sequence numbers for each table/view before/after updating. Each table/view row would be individually locked for their change (metadata for a view or table cannot span regions due to our split policy), with the sequence number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)