phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-4227) Row for "SYSTEM" schema is not created during ConnectionQueryServicesImpl init
Date Sun, 24 Sep 2017 17:49:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178294#comment-16178294
] 

Karan Mehta edited comment on PHOENIX-4227 at 9/24/17 5:48 PM:
---------------------------------------------------------------

I am referring to {{ConnectionQueryServicesImpl#init(String url, Properties props)}} method,
which first checks if SYSTEM tables are upgraded with the method {{ensureSystemTablesUpgraded()}},
which inturn checks if SYSTEM namespace is created or not by calling the method {{ensureNamespaceCreated(String
schemaName)}}. This method directly creates a HBase namespace with the HBase API, thus bypassing
its entry in the {{SYSTEM.CATALOG}} table. 
Here is the relevant piece of code.
{code}
        try (HBaseAdmin admin = getAdmin()) {
            NamespaceDescriptor namespaceDescriptor = null;
            try {
                namespaceDescriptor = admin.getNamespaceDescriptor(schemaName);
            } catch (org.apache.hadoop.hbase.NamespaceNotFoundException e) {

            }
            if (namespaceDescriptor == null) {
                namespaceDescriptor = NamespaceDescriptor.create(schemaName).build();
                admin.createNamespace(namespaceDescriptor);
            }
            return;
        } catch (IOException e) {
            sqlE = ServerUtil.parseServerException(e);
        } finally {
            if (sqlE != null) { throw sqlE; }
        }
{code}

I added the method in {{SystemTablePermissionsIT#testNamespaceMappedSystemTables()}} test
to read SYSCAT. However I found that row missing. After executing the statement {{CREATE SCHEMA
IF NOT EXISTS SYSTEM}}, the row was created in SYSCAT.
{code}
    private void readTable(String tableName) throws SQLException, IOException {
        System.out.println("Reading table: " + tableName);
        try (Connection conn = DriverManager.getConnection(getJdbcUrl(), clientProperties);)
{
            HTable table = (HTable) conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName.getBytes());
            Scan scan = new Scan();
            ResultScanner results = table.getScanner(scan);
            Result result = results.next();
            while(result != null) {
                System.out.println("TableName = " + Bytes.toString(result.getRow()));
                NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>>
map = result.getMap();
                for(Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>>
entry : map.entrySet()) {
                    for(Map.Entry<byte[], NavigableMap<Long, byte[]>> entry1 :
entry.getValue().entrySet()) {
                        System.out.println("\tentry1.getKey() = " + Bytes.toString(entry1.getKey()));
                        for(Map.Entry<Long, byte[]> entry2 : entry1.getValue().entrySet())
{
                            System.out.println("\t\tentry2.getValue() = " + Bytes.toString(entry2.getValue()));
                        }
                    }
                }
                result = results.next();
            }
        }
        System.out.println("End of reading");
    }
{code}

This init method is used by PhoenixDriver and PhoenixEmbeddedDriver, connect() method.



was (Author: karanmehta93):
I am referring to {{ConnectionQueryServicesImpl#init(String url, Properties props)}} method,
which first checks if SYSTEM tables are upgraded with the method {{ensureSystemTablesUpgraded()}},
which inturn checks if SYSTEM namespace is created or not by calling the method {{ensureNamespaceCreated(String
schemaName)}}. This method directly creates a HBase namespace with the HBase API, thus bypassing
its entry in the {{SYSTEM.CATALOG}} table. 
Here is the relevant piece of code.
{code}
        try (HBaseAdmin admin = getAdmin()) {
            NamespaceDescriptor namespaceDescriptor = null;
            try {
                namespaceDescriptor = admin.getNamespaceDescriptor(schemaName);
            } catch (org.apache.hadoop.hbase.NamespaceNotFoundException e) {

            }
            if (namespaceDescriptor == null) {
                namespaceDescriptor = NamespaceDescriptor.create(schemaName).build();
                admin.createNamespace(namespaceDescriptor);
            }
            return;
        } catch (IOException e) {
            sqlE = ServerUtil.parseServerException(e);
        } finally {
            if (sqlE != null) { throw sqlE; }
        }
{code}

I added the method in {{SystemTablePermissionsIT#testNamespaceMappedSystemTables() }} test
to read SYSCAT. However I found that row missing. After executing the statement {{CREATE SCHEMA
IF NOT EXISTS SYSTEM}}, the row was created in SYSCAT.
{code}
    private void readTable(String tableName) throws SQLException, IOException {
        System.out.println("Reading table: " + tableName);
        try (Connection conn = DriverManager.getConnection(getJdbcUrl(), clientProperties);)
{
            HTable table = (HTable) conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName.getBytes());
            Scan scan = new Scan();
            ResultScanner results = table.getScanner(scan);
            Result result = results.next();
            while(result != null) {
                System.out.println("TableName = " + Bytes.toString(result.getRow()));
                NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>>
map = result.getMap();
                for(Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>>
entry : map.entrySet()) {
                    for(Map.Entry<byte[], NavigableMap<Long, byte[]>> entry1 :
entry.getValue().entrySet()) {
                        System.out.println("\tentry1.getKey() = " + Bytes.toString(entry1.getKey()));
                        for(Map.Entry<Long, byte[]> entry2 : entry1.getValue().entrySet())
{
                            System.out.println("\t\tentry2.getValue() = " + Bytes.toString(entry2.getValue()));
                        }
                    }
                }
                result = results.next();
            }
        }
        System.out.println("End of reading");
    }
{code}

This init method is used by PhoenixDriver and PhoenixEmbeddedDriver, connect() method.


> Row for "SYSTEM" schema is not created during ConnectionQueryServicesImpl init 
> -------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4227
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4227
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.12.0
>            Reporter: Karan Mehta
>
> For every new schema created through Phoenix, a row containing the schema name (with
a 0x0 at the beginning and the end) is added to SYSTEM.CATALOG table. This is used for by
other statements such as "USE SCHEMA" or "CREATE SCHEMA" to determine if the schema exists
or not. 
> However when we turn on namespaces, SYSTEM tables are automatically migrated to SYSTEM
namespace in HBase. However an SYSTEM.CATALOG entry with the row (\x0SYSTEM\x0) is not created.
Thus other statements trying to use it might receive a SCHEMA_NOT_FOUND_EXCEPTION even though
the schema exists in the file.
> A "CREATE SCHEMA SYSTEM" statement can create the corresponding row entry in the SYSCAT
table. This JIRA is to fix this behaviour and make it consistent for all schemas across Phoenix.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message