phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
Date Wed, 05 Apr 2017 23:38:41 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958021#comment-15958021
] 

James Taylor edited comment on PHOENIX-3534 at 4/5/17 11:38 PM:
----------------------------------------------------------------

How about a new type of link row? That way we can read them independently, they'd get deleted
when the table gets dropped and they wouldn't interfere with the actual columns.

One corner case - if a base column is dropped in a view, then the base column is dropped in
the base table, and then the column is re-added to the base table. It'll stay dropped in the
view. If we want to deal with this, here are some ideas to solve it:
- check the timestamp of our new drop column marker. Only filter the column from the base
table if the timestamp of our drop column marker is newer than the timestamp of the column.
We'd need to capture the timestamp of the column and store it in our PColumn (which we don't
do today).
- convert the system catalog to use column encoding. In this case, we'd use the cq to identify
the dropped column. Since we generate a new cq with each new column, the new column would
correctly show up in the view.

One other issue that we haven't dealt with - what if the link row keys collide? We should
probably encode the link type in the row key. Or maybe have a unique cq for each link type.
Something to think about.


was (Author: jamestaylor):
How about a new type of link row? That way we can read them independently, they'd get deleted
when the table gets dropped and they wouldn't interfere with the actual columns.

One corner case - if a base column is dropped in a view, then the base column is dropped in
the base table, and then the column is re-added to the base table. It'll stay dropped in the
view. If we want to deal with this, here are some ideas to solve it:
- check the timestamp of our new drop column marker. Only filter the column from the base
table if the timestamp of our drop column marker is newer than the timestamp of the column.
We'd need to capture the timestamp of the column and store it in our PColumn (which we don't
do today).
- convert the system catalog to use column encoding. In this case, we'd use the cq to identify
the dropped column. Since we generate a new cq with each new column, the new column would
correctly show up in the view.

> Support multi region SYSTEM.CATALOG table
> -----------------------------------------
>
>                 Key: PHOENIX-3534
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3534
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: churro morales
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region based on the
server-side row locks being held for operations that impact a table and all of it's views.
For example, adding/removing a column from a base table pushes this change to all views.
> As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), when a new
table is created we can do a lazy cleanup  of any rows that may be left over from a failed
DDL call (kudos to [~lhofhansl] for coming up with this idea). To implement this efficiently,
we'd need to also do PHOENIX-2051 so that we can efficiently find derived views.
> The implementation would rely on an optimistic concurrency model based on checking our
sequence numbers for each table/view before/after updating. Each table/view row would be individually
locked for their change (metadata for a view or table cannot span regions due to our split
policy), with the sequence number being incremented under lock and then returned to the client.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message