phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
Date Fri, 02 Jun 2017 04:47:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034126#comment-16034126
] 

James Taylor edited comment on PHOENIX-3823 at 6/2/17 4:46 AM:
---------------------------------------------------------------

Thanks for the revised patch, [~sukunaidu@gmail.com]. Changes look good, but a couple more
minor items:
- Good idea to just pass through TableName to the static FromCompiler methods, but you already
have the TableName in UpsertCompiler without creating another one by doing an upsert.getTable().getName().
- Setting DROP_METADATA_ATTRIB in the doSetup method will make that test suite extremely slow.
Can we only do that for the test that needs it? Let's add a comment too about why we need
that (I can't remember why already).
- No need to explicitly drop tables because each test creates new tables.
{code}
+        } finally {
+            conn1.createStatement().execute("drop table if exists "+tableName);
{code}
- Please use the generateUniqueName() to generate the table name rather than the current time.
{code}
+        String tableName = "UpdateCacheUpsertSelectTest".toUpperCase()+System.currentTimeMillis();
{code}
- Also, can you add a comment explaining this bit of code (i.e. the necessity of using the
real PhoenixDriver to have more than one ConnectionQueryServices created so that new metadata
doesn't get propagated in from the client making the DDL call)?
{code}
+        longRunningProps.put(QueryServices.EXTRA_JDBC_ARGUMENTS_ATTRIB,
+            QueryServicesOptions.DEFAULT_EXTRA_JDBC_ARGUMENTS);
+        longRunningProps.put(QueryServices.DROP_METADATA_ATTRIB, Boolean.TRUE.toString());
+        Connection conn1 = DriverManager.getConnection(getUrl(), longRunningProps);
+        String url2 = getUrl() + PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + "LongRunningQueries";
+        Connection conn2 = DriverManager.getConnection(url2, longRunningProps);
{code}

You might need to rebase the patch too as it the build-bot is complaining it can't be applied.
Please confirm your other patches are still good to go too - I'd like to commit them all together.


was (Author: jamestaylor):
Thanks for the revised patch, [~sukunaidu@gmail.com]. Changes look good, but a couple more
minor items:
- Good idea to just pass through TableName to the static FromCompiler methods, but you already
have the TableName in UpsertCompiler without creating another one by doing an upsert.getTable().getName().
- Setting DROP_METADATA_ATTRIB in the doSetup method will make that test suite extremely slow.
Can we only do that for the test that needs it? Let's add a comment too about why we need
that (I can't remember why already).
- Also, can you add a comment explaining this bit of code (i.e. the necessity of using the
real PhoenixDriver to have more than one ConnectionQueryServices created so that new metadata
doesn't get propagated in from the client making the DDL call)?
{code}
+        longRunningProps.put(QueryServices.EXTRA_JDBC_ARGUMENTS_ATTRIB,
+            QueryServicesOptions.DEFAULT_EXTRA_JDBC_ARGUMENTS);
+        longRunningProps.put(QueryServices.DROP_METADATA_ATTRIB, Boolean.TRUE.toString());
+        Connection conn1 = DriverManager.getConnection(getUrl(), longRunningProps);
+        String url2 = getUrl() + PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + "LongRunningQueries";
+        Connection conn2 = DriverManager.getConnection(url2, longRunningProps);
{code}

You might need to rebase the patch too as it the build-bot is complaining it can't be applied.
Please confirm your other patches are still good to go too - I'd like to commit them all together.

> Force cache update on MetaDataEntityNotFoundException 
> ------------------------------------------------------
>
>                 Key: PHOENIX-3823
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3823
>             Project: Phoenix
>          Issue Type: Sub-task
>    Affects Versions: 4.10.0
>            Reporter: James Taylor
>            Assignee: Maddineni Sukumar
>             Fix For: 4.11.0
>
>         Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, PHOENIX-3823.v3.patch,
PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch,
PHOENIX-3823.v8.patch, PHOENIX-3823.v9.patch
>
>
> When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period of time
which may cause the schema being used to become stale. If another client adds a column or
a new table or view, other clients won't see it. As a result, the client will get a MetaDataEntityNotFoundException.
Instead of bubbling this up, we should retry after forcing a cache update on the tables involved
in the query.
> The above works well for references to entities that don't yet exist. However, we cannot
detect when some entities are referred to which no longer exists until the cache expires.
An exception is if a physical table is dropped which would be detected immediately, however
we would allow queries and updates to columns which have been dropped until the cache entry
expires (which seems like a reasonable tradeoff IMHO. In addition, we won't start using indexes
on tables until the cache expires.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message