phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-2862) Do client server compatibility checks before upgrading system tables
Date Fri, 06 May 2016 15:49:13 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274249#comment-15274249
] 

James Taylor edited comment on PHOENIX-2862 at 5/6/16 3:49 PM:
---------------------------------------------------------------

I agree that you'd want to prevent a 4.7 client from connecting to a 4.8 server once the system
tables have been migrated to namespaces, [~ankit.singhal]. This is tricky, as you need to
detect this on the server-side, as the client won't be different. It seems like this patch
is changing the client-side to detect this situation which won't help.

How about this approach?
- In MetaDataEndPointImpl.createTable, if client is earlier than 4.8 and we're attempting
to create the SYSTEM.CATALOG table, check if the CATALOG HBase table already exists in the
SYSTEM namespace. If it does, set the return code to UNALLOWED_TABLE_MUTATION and return.
{code}
    @Override
    public void createTable(RpcController controller, CreateTableRequest request,
            RpcCallback<MetaDataResponse> done) {
        MetaDataResponse.Builder builder = MetaDataResponse.newBuilder();
        byte[][] rowKeyMetaData = new byte[3][];
        byte[] schemaName = null;
        byte[] tableName = null;
        try {
            List<Mutation> tableMetadata = ProtobufUtil.getMutations(request);
            MetaDataUtil.getTenantIdAndSchemaAndTableName(tableMetadata, rowKeyMetaData);
            byte[] tenantIdBytes = rowKeyMetaData[PhoenixDatabaseMetaData.TENANT_ID_INDEX];
            schemaName = rowKeyMetaData[PhoenixDatabaseMetaData.SCHEMA_NAME_INDEX];
            tableName = rowKeyMetaData[PhoenixDatabaseMetaData.TABLE_NAME_INDEX];

            byte[] parentSchemaName = null;
            byte[] parentTableName = null;
            PTableType tableType = MetaDataUtil.getTableType(tableMetadata, GenericKeyValueBuilder.INSTANCE,
new ImmutableBytesWritable());
            byte[] parentTableKey = null;
            Mutation viewPhysicalTableRow = null;
            if (tableType == PTableType.VIEW) {
                byte[][] parentSchemaTableNames = new byte[2][];
                /*
                 * For a view, we lock the base physical table row. For a mapped view, there
is 
                 * no link present to the physical table. So the viewPhysicalTableRow is null
                 * in that case.
                 */
                viewPhysicalTableRow = getPhysicalTableForView(tableMetadata, parentSchemaTableNames);
                parentSchemaName = parentSchemaTableNames[0];
                parentTableName = parentSchemaTableNames[1];
                if (parentTableName != null) {
                    parentTableKey = SchemaUtil.getTableKey(ByteUtil.EMPTY_BYTE_ARRAY, parentSchemaName,
parentTableName);
                }
            } else if (tableType == PTableType.INDEX) {
                parentSchemaName = schemaName;
                /* 
                 * For an index we lock the parent table's row which could be a physical table
or a view.
                 * If the parent table is a physical table, then the tenantIdBytes is empty
because
                 * we allow creating an index with a tenant connection only if the parent
table is a view.
                 */ 
                parentTableName = MetaDataUtil.getParentTableName(tableMetadata);
                parentTableKey = SchemaUtil.getTableKey(tenantIdBytes, parentSchemaName, parentTableName);
            } else {
                // TODO: If client is earlier than 4.8 and we're attempting to create the
SYSTEM.CATALOG table,
                // check if the CATALOG HBase table already exists in the SYSTEM namespace.
If it does, set the
                // return code to UNALLOWED_TABLE_MUTATION and return.
                if ((!request.hasClientVersion() // pre-dates sending client version
                    || request.getClientVersion() < VersionUtil.encodeVersion(4, 8, 0))
                    && table being created is SYSTEM.CATALOG) ...
            }
{code}

I don't think we can prevent the creation of the HBase metadata for the SYSTEM.CATALOG table,
but I don't think that'll cause any problems. We just want to prevent the SYSTEM.CATALOG row
from being put into the SYSTEM.CATALOG table when it's already been migrated.


was (Author: jamestaylor):
I agree that you'd want to prevent a 4.7 client from connecting to a 4.8 server once the system
tables have been migrated to namespaces, [~ankit.singhal]. This is tricky, as you need to
detect this on the server-side, as the client won't be different. It seems like this patch
is changing the client-side to detect this situation which won't help.

How about this approach?
- In MetaDataEndPointImpl.createTable, if client is earlier than 4.8 and we're attempting
to create the SYSTEM.CATALOG table, check if the CATALOG HBase table already exists in the
SYSTEM namespace. If it does, set the return code to UNALLOWED_TABLE_MUTATION and return.
{code}
    @Override
    public void createTable(RpcController controller, CreateTableRequest request,
            RpcCallback<MetaDataResponse> done) {
        MetaDataResponse.Builder builder = MetaDataResponse.newBuilder();
        byte[][] rowKeyMetaData = new byte[3][];
        byte[] schemaName = null;
        byte[] tableName = null;
        try {
            List<Mutation> tableMetadata = ProtobufUtil.getMutations(request);
            MetaDataUtil.getTenantIdAndSchemaAndTableName(tableMetadata, rowKeyMetaData);
            byte[] tenantIdBytes = rowKeyMetaData[PhoenixDatabaseMetaData.TENANT_ID_INDEX];
            schemaName = rowKeyMetaData[PhoenixDatabaseMetaData.SCHEMA_NAME_INDEX];
            tableName = rowKeyMetaData[PhoenixDatabaseMetaData.TABLE_NAME_INDEX];

            byte[] parentSchemaName = null;
            byte[] parentTableName = null;
            PTableType tableType = MetaDataUtil.getTableType(tableMetadata, GenericKeyValueBuilder.INSTANCE,
new ImmutableBytesWritable());
            byte[] parentTableKey = null;
            Mutation viewPhysicalTableRow = null;
            if (tableType == PTableType.VIEW) {
                byte[][] parentSchemaTableNames = new byte[2][];
                /*
                 * For a view, we lock the base physical table row. For a mapped view, there
is 
                 * no link present to the physical table. So the viewPhysicalTableRow is null
                 * in that case.
                 */
                viewPhysicalTableRow = getPhysicalTableForView(tableMetadata, parentSchemaTableNames);
                parentSchemaName = parentSchemaTableNames[0];
                parentTableName = parentSchemaTableNames[1];
                if (parentTableName != null) {
                    parentTableKey = SchemaUtil.getTableKey(ByteUtil.EMPTY_BYTE_ARRAY, parentSchemaName,
parentTableName);
                }
            } else if (tableType == PTableType.INDEX) {
                parentSchemaName = schemaName;
                /* 
                 * For an index we lock the parent table's row which could be a physical table
or a view.
                 * If the parent table is a physical table, then the tenantIdBytes is empty
because
                 * we allow creating an index with a tenant connection only if the parent
table is a view.
                 */ 
                parentTableName = MetaDataUtil.getParentTableName(tableMetadata);
                parentTableKey = SchemaUtil.getTableKey(tenantIdBytes, parentSchemaName, parentTableName);
            } else {
                // TODO: If client is earlier than 4.8 and we're attempting to create the
SYSTEM.CATALOG table,
                // check if the CATALOG HBase table already exists in the SYSTEM namespace.
If it does, set the
                // return code to UNALLOWED_TABLE_MUTATION and return.
                if ((!request.hasClientVersion() // pre-dates sending client version
                    || request.getClientVersion() < VersionUtil.encodeVersion(4, 8, 0))
                    && table being created is SYSTEM.CATALOG) ...
            }

I don't think we can prevent the creation of the HBase metadata for the SYSTEM.CATALOG table,
but I don't think that'll cause any problems. We just want to prevent the SYSTEM.CATALOG row
from being put into the SYSTEM.CATALOG table when it's already been migrated.

> Do client server compatibility checks before upgrading system tables
> --------------------------------------------------------------------
>
>                 Key: PHOENIX-2862
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2862
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: Ankit Singhal
>            Assignee: Ankit Singhal
>             Fix For: 4.8.0
>
>         Attachments: PHOENIX-2862.patch
>
>
> currently , we allow upgrade of system tables to map to system namespace by enabling
"phoenix.schema.mapSystemTablesToNamespace" config (conjuction with "phoenix.connection.isNamespaceMappingEnabled")

> but we need to ensure following things whenever client connects with above config:-
> 1. Server should be upgraded and check consistency of these properties between client
and server.
> 2. If above property does not exists but system:catalog exists, we should not start creating
system.catalog.
> 3. if old client connects, it should not create system.catalog again ignoring the upgrade
and start using it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message