phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4523) phoenix.schema.isNamespaceMappingEnabled problem
Date Wed, 10 Jan 2018 19:10:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320877#comment-16320877
] 

Karan Mehta commented on PHOENIX-4523:
--------------------------------------

[~f.pompermaier] Thank you for the responses. I have a potential patch ready but want to confirm
with few more questions.

bq. I debugged a little more the code and it seems that the ensureSystemTablesMigratedToSystemNamespace()
is always called, causing a lock to be created.

That method is always called, however if SYSTEM tables are already migrated to SYSTEM namespace,
then {{getSystemTableNames()}} will not return any tables and hence the method would return
from that point. Even if it did go inside, having a row with the value {{UPGRADE_MUTEX_UNLOCKED}}
means that the lock was not acquired. Lock is only acquired during SYSTEM.CATALOG upgrade
or SYSTEM table migration. If both of them are done, then it never be acquired again.

This stack trace still doesn't tell where exactly the problem started from. I checked out
the tag {{v4.13.1-HBase-1.2-rc0}} and the only place {{createSysMutexTable()}} method is called
is inside {{createOtherSystemTables()}} method. The master branch has multiple callers to
this method. Do you want me to test with master branch?

bq. it seems that TableExistsException is not catched within createSysMutexTable() because
is wrapped by a org.apache.hadoop.ipc.RemoteException, and here I don't know why that exception
is not wrapped during local debug

This and the check for SYSMUTEX table seems to the valid issue. I will fix it in the patch.

bq. So it appears that in the case of a cluster that hasn't been upgraded yet, there are problems
with the execution of the ensureSystemTablesMigratedToSystemNamespace(). 

According to [~f.pompermaier], the cluster is already migrated. So the {{ensureSystemTablesMigratedToSystemNamespace()}}
method is almost short-circuited.

bq. can we do a little refactoring and call a new method instead of createSysMutexTable that
doesn't again check for the existence of SYSTEM.MUTEX (since we already check before calling
createSysMutexTable

I will do some refactoring too.

bq. why in createSysMutexTable do we call admin.listTableNames() and then one line later call
admin.tableExists()? That seems like a complete waste.

We should not worry much about optimization here IMHO, since this code path is not frequently
executed. The code is already messy and trying to optimize might result into some unexpected
issues.

I will put up a patch once these things are confirmed. 


> phoenix.schema.isNamespaceMappingEnabled problem
> ------------------------------------------------
>
>                 Key: PHOENIX-4523
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4523
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.13.1
>            Reporter: Flavio Pompermaier
>            Assignee: Karan Mehta
>
> I'm using Phoenix 4.13 for CDH 5.11.2 parcel and enabling schemas made my code unusable.
> I think that this is not a bug of the CDH release, but of all 4.13.x releases.
> I have many parallel Phoenix connections and I always get the following exception:
> {code:java}
> Caused by: java.sql.SQLException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.TableExistsException):
SYSTEM:MUTEX
> 	at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2492)
> 	at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2384)
> 	at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
> 	at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2384)
> 	at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
> 	at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
> 	at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> 	at java.sql.DriverManager.getConnection(DriverManager.java:664)
> 	at java.sql.DriverManager.getConnection(DriverManager.java:270)
> {code}
> This is caused by the fact that all the times the SYSTEM tables are recreated, and this
cannot be done simultaneously.
> Trying to debug the issue I found that in ConnectionQueryServicesImpl.createSysMutexTable()
the call to getSystemTableNames() always return an empty array and the SYSTEM:MUTEX  table
is always recreated.
> This because getSystemTableNames() doesn't consider the case when system tables have
namespace enabled. Right now that method tries to get all tables starting with *SYSTEM.\**,
while it should try to get the list of *SYSTEM:\** tables..
> I hope this could get fixed very soon,
> Flavio



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message