hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Dere <jd...@hortonworks.com>
Subject Re: Review Request 52171: HIVE-14819 FunctionInfo for permanent functions shows TEMPORARY FunctionType
Date Fri, 23 Sep 2016 21:47:17 GMT


> On Sept. 22, 2016, 11:53 p.m., Sergey Shelukhin wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java, line 499
> > <https://reviews.apache.org/r/52171/diff/1/?file=1508429#file1508429line499>
> >
> >     why shouldn't the just be in one?
> 
> Jason Dere wrote:
>     That's been the behavior since HIVE-2573 - permanent UDFs loaded at Hive initialization
are put in the system registry, and added to the session registry once they are used by that
session.
>     I think they at least need to go to the session registry, it's also useful since
it can let us know that the UDF resources have already been loaded for that session. If we
completely remove the UDF loading behavior in Hive initialization we might be able to remove
permanent UDFs from the system registry. But you kept this behavior around in HIVE-13596!
:)

Looking at this a bit more, the initial permanent UDF loading mechanism in Hive.reloadFunctions()
is actually broken a bit by HIVE-12857 - it's trying to instantiate the UDF class and add
it to Registry.persistent. But reloadFunctions() was never intended to actually load all of
the UDF jars, so it fails to instantiate the class and reloadFunctions() actually fails to
register the function during reloadFunctions(). I'll add a fix to persistent function detection
related to this issue.
We can open a followup issue to revisit whether permanent functions should be loaded during
Hive initialization.


- Jason


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52171/#review150102
-----------------------------------------------------------


On Sept. 22, 2016, 7:21 p.m., Jason Dere wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52171/
> -----------------------------------------------------------
> 
> (Updated Sept. 22, 2016, 7:21 p.m.)
> 
> 
> Review request for hive and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-14819
>     https://issues.apache.org/jira/browse/HIVE-14819
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Patch to allow the registry to set PERSISTENT type when registering permanent functions
to the session registry. Previously all functions added to session registry had the TEMPORARY
tag
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionInfo.java 30ba996 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java 05926b5 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/WindowFunctionInfo.java a16d9e5 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java
911b86b 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestFunctionRegistry.java d2d5a1b 
> 
> Diff: https://reviews.apache.org/r/52171/diff/
> 
> 
> Testing
> -------
> 
> Added tests to TestFunctionRegistry
> 
> 
> Thanks,
> 
> Jason Dere
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message