hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kongxianghe (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17462) hive_1.2.1 hiveserver2 memory leak
Date Mon, 21 Sep 2020 07:12:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199204#comment-17199204
] 

kongxianghe commented on HIVE-17462:
------------------------------------

was fixed in https://issues.apache.org/jira/browse/HIVE-10453
you may use hdp-1.2.1000.2.6.1.0 or some other versions,this patch may be not added into it.
decompile your hive-exec-1.2.1xxx.jar  and find class SessionState.class .

{code}
public void close() throws IOException {
    registry.clear();;
    registry.clear();
    if (txnMgr != null) txnMgr.closeTxnManager();
    JavaUtils.closeClassLoadersTo(conf.getClassLoader(), parentLoader);
    File resourceDir =
@@ -1493,7 +1493,7 @@ public class SessionState {
        sparkSession = null;
      }
    }

    // this line might be lost!!
    registry.closeCUDFLoaders();
    dropSessionPaths(conf);
  }
{code}
registry.closeCUDFLoaders();
in hdp 2.6.1  hive-1.2.1  this line is missing,might clause this problem 

> hive_1.2.1  hiveserver2  memory leak
> ------------------------------------
>
>                 Key: HIVE-17462
>                 URL: https://issues.apache.org/jira/browse/HIVE-17462
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.1
>         Environment: hive  version  1.2.1
>            Reporter: gehaijiang
>            Priority: Major
>
> hiveserver2  memory leak
> hive  use  third UDF  (vs-1.0.2-SNAPSHOT.jar , alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
 ..... and so on )
> lr-x------ 1 data data 64 Sep  5 18:37 964 -> /tmp/9e38cc04-5693-474b-9c7d-bfdd978bcbb4_resources/vs-1.0.2-SNAPSHOT.jar
(deleted)
> lr-x------ 1 data data 64 Sep  6 10:41 965 -> /tmp/188bbf2a-d8a5-48a7-81fc-b807f9ff201d_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
(deleted)
> lr-x------ 1 data data 64 Sep  6 17:41 97 -> /home/data/programs/hadoop-2.7.1/share/hadoop/hdfs/lib/jsr305-3.0.0.jar
> lrwx------ 1 data data 64 Sep  5 18:37 975 -> socket:[1318353317]
> lr-x------ 1 data data 64 Sep  6 02:38 977 -> /tmp/64e309dc-352f-4ba4-b871-1aa78fe05945_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
(deleted)
> lr-x------ 1 data data 64 Sep  6 17:41 98 -> /home/data/programs/hadoop-2.7.1/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar
> lrwx------ 1 data data 64 Sep  6 08:40 983 -> socket:[1299459344]
> lr-x------ 1 data data 64 Sep  5 19:37 987 -> /tmp/c3054987-c9c6-468a-8b5c-6e20b1972e0b_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
(deleted)
> lr-x------ 1 data data 64 Sep  6 17:41 99 -> /home/data/programs/hadoop-2.7.1/share/hadoop/hdfs/lib/guava-11.0.2.jar
> lr-x------ 1 data data 64 Sep  6 08:40 994 -> /tmp/fc5c44b3-9bd8-4a32-a39a-66cd44032fee_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
(deleted)
> lr-x------ 1 data data 64 Sep  6 06:39 996 -> /tmp/3b3c2bd6-0a0e-4599-b757-4a048a968457_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
(deleted)
> lr-x------ 1 data data 64 Sep  5 17:36 999 -> /tmp/6ad76494-cdda-430b-b7d0-2213731655a8_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar
(deleted)
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 20084 data      20   0 13.6g  11g 533m S 62.3  9.2   6619:16 java
> /home/data/programs/jdk/jdk-current/bin/java-Djava.net.preferIPv4Stack=true-Dhadoop.log.dir=/home/data/hadoop/logs-Dhadoop.log.file=hadoop.log-Dhadoop.home.dir=/home/data/programs/hadoop-2.7.1-Dhadoop.id.str=data-Dhadoop.root.logger=INFO,DRFA-Djava.library.path=/home/data/programs/hadoop-2.7.1/lib/native-Dhadoop.policy.file=hadoop-policy.xml-Djava.net.preferIPv4Stack=true-XX:+UseConcMarkSweepGC-Xms8g-Xmx8g-Dhadoop.security.logger=INFO,NullAppenderorg.apache.hadoop.util.RunJar/home/data/programs/hive-current/lib/hive-service-1.2.1.jarorg.apache.hive.service.server.HiveServer2--hiveconfhive.log.file=hiveserver2.log



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message