hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
Date Tue, 26 Apr 2016 19:12:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258724#comment-15258724
] 

Sergey Shelukhin commented on HIVE-13596:
-----------------------------------------

{noformat}
    Should this be a settable option (as opposed to always on)? And why default to false?
{noformat}
It's settable per session. Defaulting to false because that was the current behavior for a
while.

{noformat}
    Which Registry is performing the UDF lookup, the system registry or the session registry?
If it is the system registry, then we may run into HIVE-6672 again. checkFunctionClass() (removed
in your patch) was added for this purpose.
{noformat}
Session registry. I removed the method because it was unused... 

{noformat}
    If the functions are being looked up/added to the session registry, then this may not
be an issue because every session would need to lookup UDF/load JARs. Actually I see that
the permanent UDFs by Hive.reloadFunctions() (at initialize time) are added to the system
registry .. I suspect Hive probably has class loading issues if we ever use "RELOAD FUNCTIONS"
to pick up new UDFs, since Hive no longer seems to be calling checkFunctionClass().
{noformat}
Hmm... should this be done in the system registry then? Does it hurt to have them in the session
registry? 
Also, does checkFunctionClass need to be reinstated here or in a separate JIRA.

{noformat}
    public Registry(boolean isNative, HiveConf conf):
    conf needs a null check before it's used
{noformat}
Implied that it's not used for a native registry; I will add an explicit check.


> HS2 should be able to get UDFs on demand from metastore
> -------------------------------------------------------
>
>                 Key: HIVE-13596
>                 URL: https://issues.apache.org/jira/browse/HIVE-13596
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-13596.patch
>
>
> When multiple HS2s are run, creating a permanent fn is only executed on one of them,
and the other HS2s don't get the new function. Unlike say with tables, where we always get
stuff from db on demand, fns are registered at certain points in the code and if the new one
is not registered, it will not be available. 
> We should restore the pre-HIVE-2573 behavior of being able to refresh the UDFs on demand.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message