From issues-return-199270-archive-asf-public=cust-asf.ponee.io@hive.apache.org Mon Sep 21 07:12:03 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 3E633180607 for ; Mon, 21 Sep 2020 09:12:03 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 785F84277D for ; Mon, 21 Sep 2020 07:12:02 +0000 (UTC) Received: (qmail 26012 invoked by uid 500); 21 Sep 2020 07:12:02 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 26002 invoked by uid 99); 21 Sep 2020 07:12:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Sep 2020 07:12:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 3417042A11 for ; Mon, 21 Sep 2020 07:12:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 81FD67802B1 for ; Mon, 21 Sep 2020 07:12:00 +0000 (UTC) Date: Mon, 21 Sep 2020 07:12:00 +0000 (UTC) From: "kongxianghe (Jira)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-17462) hive_1.2.1 hiveserver2 memory leak MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-17462?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1719= 9204#comment-17199204 ]=20 kongxianghe commented on HIVE-17462: ------------------------------------ was fixed in https://issues.apache.org/jira/browse/HIVE-10453 you may use hdp-1.2.1000.2.6.1.0 or some other versions,this patch may be n= ot added into it. decompile your hive-exec-1.2.1xxx.jar and find class SessionState.class . {code} public void close() throws IOException { registry.clear();; registry.clear(); if (txnMgr !=3D null) txnMgr.closeTxnManager(); JavaUtils.closeClassLoadersTo(conf.getClassLoader(), parentLoader); File resourceDir =3D @@ -1493,7 +1493,7 @@ public class SessionState { sparkSession =3D null; } } // this line might be lost!! registry.closeCUDFLoaders(); dropSessionPaths(conf); } {code} registry.closeCUDFLoaders(); in hdp 2.6.1 hive-1.2.1 this line is missing,might clause this problem=20 > hive_1.2.1 hiveserver2 memory leak > ------------------------------------ > > Key: HIVE-17462 > URL: https://issues.apache.org/jira/browse/HIVE-17462 > Project: Hive > Issue Type: Bug > Affects Versions: 1.2.1 > Environment: hive version 1.2.1 > Reporter: gehaijiang > Priority: Major > > hiveserver2 memory leak > hive use third UDF =EF=BC=88vs-1.0.2-SNAPSHOT.jar =EF=BC=8C alogdata-1= .0.3-SNAPSHOT-jar-with-dependencies.jar ..... and so on ) > lr-x------ 1 data data 64 Sep 5 18:37 964 -> /tmp/9e38cc04-5693-474b-9c7= d-bfdd978bcbb4_resources/vs-1.0.2-SNAPSHOT.jar (deleted) > lr-x------ 1 data data 64 Sep 6 10:41 965 -> /tmp/188bbf2a-d8a5-48a7-81f= c-b807f9ff201d_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar = (deleted) > lr-x------ 1 data data 64 Sep 6 17:41 97 -> /home/data/programs/hadoop-2= .7.1/share/hadoop/hdfs/lib/jsr305-3.0.0.jar > lrwx------ 1 data data 64 Sep 5 18:37 975 -> socket:[1318353317] > lr-x------ 1 data data 64 Sep 6 02:38 977 -> /tmp/64e309dc-352f-4ba4-b87= 1-1aa78fe05945_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar = (deleted) > lr-x------ 1 data data 64 Sep 6 17:41 98 -> /home/data/programs/hadoop-2= .7.1/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar > lrwx------ 1 data data 64 Sep 6 08:40 983 -> socket:[1299459344] > lr-x------ 1 data data 64 Sep 5 19:37 987 -> /tmp/c3054987-c9c6-468a-8b5= c-6e20b1972e0b_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar = (deleted) > lr-x------ 1 data data 64 Sep 6 17:41 99 -> /home/data/programs/hadoop-2= .7.1/share/hadoop/hdfs/lib/guava-11.0.2.jar > lr-x------ 1 data data 64 Sep 6 08:40 994 -> /tmp/fc5c44b3-9bd8-4a32-a39= a-66cd44032fee_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar = (deleted) > lr-x------ 1 data data 64 Sep 6 06:39 996 -> /tmp/3b3c2bd6-0a0e-4599-b75= 7-4a048a968457_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar = (deleted) > lr-x------ 1 data data 64 Sep 5 17:36 999 -> /tmp/6ad76494-cdda-430b-b7d= 0-2213731655a8_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar = (deleted) > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 20084 data 20 0 13.6g 11g 533m S 62.3 9.2 6619:16 java > /home/data/programs/jdk/jdk-current/bin/java-Djava.net.preferIPv4Stack=3D= true-Dhadoop.log.dir=3D/home/data/hadoop/logs-Dhadoop.log.file=3Dhadoop.log= -Dhadoop.home.dir=3D/home/data/programs/hadoop-2.7.1-Dhadoop.id.str=3Ddata-= Dhadoop.root.logger=3DINFO,DRFA-Djava.library.path=3D/home/data/programs/ha= doop-2.7.1/lib/native-Dhadoop.policy.file=3Dhadoop-policy.xml-Djava.net.pre= ferIPv4Stack=3Dtrue-XX:+UseConcMarkSweepGC-Xms8g-Xmx8g-Dhadoop.security.log= ger=3DINFO,NullAppenderorg.apache.hadoop.util.RunJar/home/data/programs/hiv= e-current/lib/hive-service-1.2.1.jarorg.apache.hive.service.server.HiveServ= er2--hiveconfhive.log.file=3Dhiveserver2.log -- This message was sent by Atlassian Jira (v8.3.4#803005)