spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gurwls...@apache.org
Subject [spark] branch master updated: [MINOR][SQL] Unnecessary access to externalCatalog
Date Thu, 11 Apr 2019 01:00:33 GMT
This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 181d190  [MINOR][SQL] Unnecessary access to externalCatalog
181d190 is described below

commit 181d190c606ec6cbd09f6d618347b60eaa4ea828
Author: ocaballero <oliver.caballero.alvarez@gmail.com>
AuthorDate: Thu Apr 11 10:00:09 2019 +0900

    [MINOR][SQL] Unnecessary access to externalCatalog
    
    Necessarily access the external catalog without having to do it
    
    ## What changes were proposed in this pull request?
    
    The existsFunction function has been changed because it unnecessarily accessed the externalCatalog
to find if the database exists in cases where the function is in the functionRegistry
    
    ## How was this patch tested?
    
    It has been tested through spark-shell and accessing the metastore logs of hive.
    
    Inside spark-shell we use spark.table (% tableA%). SelectExpr ("trim (% columnA%)") in
the current version and it appears every time:
    
    org.apache.hadoop.hive.metastore.HiveMetaStore.audit: cmd = get_database: default
    
    Once the change is made, no record appears
    
    Closes #24312 from OCaballero/master.
    
    Authored-by: ocaballero <oliver.caballero.alvarez@gmail.com>
    Signed-off-by: HyukjinKwon <gurwls223@apache.org>
---
 .../org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala     | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
index 4b862a5..c05f777 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
@@ -1105,10 +1105,11 @@ class SessionCatalog(
    * Check if the function with the specified name exists
    */
   def functionExists(name: FunctionIdentifier): Boolean = {
-    val db = formatDatabaseName(name.database.getOrElse(getCurrentDatabase))
-    requireDbExists(db)
-    functionRegistry.functionExists(name) ||
+    functionRegistry.functionExists(name) || {
+      val db = formatDatabaseName(name.database.getOrElse(getCurrentDatabase))
+      requireDbExists(db)
       externalCatalog.functionExists(db, name.funcName)
+    }
   }
 
   // ----------------------------------------------------------------


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message