hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajat Khandelwal <rajatgupt...@gmail.com>
Subject Review Request 48544: HIVE-13903: getFunctionInfo is downloading jar on every call
Date Fri, 10 Jun 2016 12:01:12 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48544/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-13903
    https://issues.apache.org/jira/browse/HIVE-13903


Repository: hive-git


Description
-------

on queries using permanent udfs, the jar file of the udf is downloaded multiple times. Each
call originating from Registry.getFunctionInfo. This increases time for the query, especially
if that query is just an explain query. The jar should be downloaded once, and not downloaded
again if the udf class is accessible in the current thread.


Diffs
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bb0ca3acf79426df5d34d44da8595ac78edf8c76

  itests/hive-unit/src/test/java/org/apache/hadoop/hive/llap/ext/TestLlapInputSplit.java 1de8aa6686a5b5db697fb4dee4b4155b9dc14ef2

  llap-client/src/java/org/apache/hadoop/hive/llap/LlapInputSplit.java 02aedfd8929038e763ed413a3545f68117e08638

  llap-client/src/java/org/apache/hadoop/hive/llap/SubmitWorkInfo.java 670429412923566536dc933463a64daabc5adb45

  llap-client/src/java/org/apache/hadoop/hive/llap/coordinator/LlapCoordinator.java PRE-CREATION

  llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java
0edb1cdd04df578228cad97cf85216186c04007d 
  llap-client/src/java/org/apache/hadoop/hive/llap/io/api/LlapProxy.java 6c2618bcd3ca9f74302c3260b1535b892f543d23

  llap-client/src/java/org/apache/hadoop/hive/llap/security/LlapTokenLocalClient.java af889b6a5af6499f01834886c5746243ad5847c1

  llap-client/src/java/org/apache/hadoop/hive/llap/security/LlapTokenLocalClientImpl.java
PRE-CREATION 
  llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java
16531e3445a9bfbd55a788674fa9f4132ca6b697 
  llap-common/src/java/org/apache/hadoop/hive/llap/NotTezEventHelper.java PRE-CREATION 
  llap-common/src/java/org/apache/hadoop/hive/llap/security/LlapSigner.java 478a40a4edded3f911df636eda655be2c0516ba3

  llap-common/src/java/org/apache/hadoop/hive/llap/security/LlapSignerImpl.java PRE-CREATION

  llap-common/src/java/org/apache/hadoop/hive/llap/security/LlapTokenLocalClient.java PRE-CREATION

  llap-common/src/java/org/apache/hadoop/hive/llap/security/SecretManager.java 540f978fd6244050688c6cc767d20716186410d3

  llap-common/src/java/org/apache/hadoop/hive/llap/security/SigningSecretManager.java 067a98eaab81a69ead61c45595da0f3b89936b4e

  llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java e43b72b5e5f7e78168fc6b2b9c48cabad12c035f

  llap-common/src/protobuf/LlapDaemonProtocol.proto 486ba0a19483663b2dec622e6c52c45b12aa8512

  llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 1df6df03ec7bc8655e085c3247718994334573b5

  llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 4306c222442546c9c54162f95e3550ea978be1b7

  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java 6f21d3cb430cb2a9e8a1b4d94b80f286602b4882

  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapTaskReporter.java dc4482eaeb875d6255a389d81e1b79d8a10a4b7a

  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java f97585d16609854a50f18b584fe3e1454de88c84

  llap-server/src/java/org/apache/hadoop/hive/llap/security/LlapSignerImpl.java 417459353e3fbe5d64ac95eaed4a37db58c17d1d

  llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java
1df5253f4654f7340f0180d36112a997fa9e51c0 
  llap-server/src/test/org/apache/hadoop/hive/llap/security/TestLlapSignerImpl.java 04202254965b6ff78fa19eb122710a9fa01f7a6f

  llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java 026df3bbfef3fe6f13d1bf9b0dc18b07e3cc817e

  orc/src/java/org/apache/orc/OrcConf.java 6fcbb72aeea40dea9d9143a334b42fb81c7f4404 
  ql/pom.xml db03978389900ff00e725cdabdf19226403f8804 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java 86df74d26ddd60736214c0f9acb835f4253a2c17

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java 54fc3af94c6b45f2e7ed8edb0b8f45d4fa0dbf2b

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java d04cfa3b102e1674bb2be2d8bebceb717dabc46a

  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 4bf2403704f4f7dd1c83ebc28f34eb01e18a0066

  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java 018d8d0d4bf35178e23453e5c6f0917e7edefc55

  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java 140dbdab5162ac865a9d6427aaf4a972cebbd827

  ql/src/java/org/apache/tez/dag/api/TaskSpecBuilder.java 6d3180200ce57eec8ce69c6fa8e98d1a9b6d7a52

  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcRecordUpdater.java 0a61fb884a8e5825e95b442d4c97efa247da4170

  ql/src/test/queries/clientpositive/windowing_range_multiorder.q d8ca4d64620d01834c24ae729332dd8d053ea1b6

  ql/src/test/results/clientpositive/tez/acid_globallimit.q.out 804e5e25a65915fb867e0ca23181aafbc791ffdc

  ql/src/test/results/clientpositive/windowing_range_multiorder.q.out 9910883680f45108bf3db59041d9b5ec6e749cdb

  service/src/java/org/apache/hive/service/server/HiveServer2.java d61edf50df576af582aa9e9fb7112f8e72224043

  shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java 68fac17311474108389fdf2dcea2baa52504ffd9


Diff: https://reviews.apache.org/r/48544/diff/


Testing
-------


Thanks,

Rajat Khandelwal


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message