phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
Date Mon, 25 Jun 2018 15:01:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522396#comment-16522396
] 

ASF GitHub Bot commented on PHOENIX-3534:
-----------------------------------------

Github user twdsilva commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/303#discussion_r197829637
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilter.java
---
    @@ -35,20 +35,18 @@
      * during cluster upgrades. However, tenant-owned data such as tenant-owned views need
to
      * be copied. This WALEntryFilter will only allow tenant-owned rows in SYSTEM.CATALOG
to
      * be replicated. Data from all other tables is automatically passed. It will also copy
    - * child links in SYSTEM.CATALOG that are globally-owned but point to tenant-owned views.
    + * child links in SYSTEM.CHILD_LINK that are globally-owned but point to tenant-owned
views.
      *
      */
     public class SystemCatalogWALEntryFilter implements WALEntryFilter {
     
    -  private static byte[] CHILD_TABLE_BYTES =
    -      new byte[]{PTable.LinkType.CHILD_TABLE.getSerializedValue()};
    -
       @Override
       public WAL.Entry filter(WAL.Entry entry) {
     
    -    //if the WAL.Entry's table isn't System.Catalog, it auto-passes this filter
    +    //if the WAL.Entry's table isn't System.Catalog or System.Child_Link, it auto-passes
this filter
         //TODO: when Phoenix drops support for pre-1.3 versions of HBase, redo as a WALCellFilter
    -    if (!SchemaUtil.isMetaTable(entry.getKey().getTablename().getName())){
    +    byte[] tableName = entry.getKey().getTablename().getName();
    +	if (!SchemaUtil.isMetaTable(tableName) && !SchemaUtil.isChildLinkTable(tableName)){
    --- End diff --
    
    SYSTEM.CHILD_LINK contains the parent->child linking rows and cells we use to detect
race conditions (eg a column of conflicting type being added at the same time to a parent
and child). 
    The latter cells are written with a short TTL. 
    I think we can use HBase replication for SYSTEM.CHILD_LINK. All the tenant specific view
metadata rows in SYSTEM.CATALOG start with tenant id. 
    I will modify this filter to how it was before PHOENIX-4229. 
    @gjacoby126  Thanks for the suggestion.


> Support multi region SYSTEM.CATALOG table
> -----------------------------------------
>
>                 Key: PHOENIX-3534
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3534
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Thomas D'Silva
>            Priority: Major
>         Attachments: PHOENIX-3534-wip.patch
>
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region based on the
server-side row locks being held for operations that impact a table and all of it's views.
For example, adding/removing a column from a base table pushes this change to all views.
> As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), when a new
table is created we can do a lazy cleanup  of any rows that may be left over from a failed
DDL call (kudos to [~lhofhansl] for coming up with this idea). To implement this efficiently,
we'd need to also do PHOENIX-2051 so that we can efficiently find derived views.
> The implementation would rely on an optimistic concurrency model based on checking our
sequence numbers for each table/view before/after updating. Each table/view row would be individually
locked for their change (metadata for a view or table cannot span regions due to our split
policy), with the sequence number being incremented under lock and then returned to the client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message