Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4C69B108E2 for ; Wed, 29 Jan 2014 21:12:17 +0000 (UTC) Received: (qmail 77230 invoked by uid 500); 29 Jan 2014 21:12:13 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 77117 invoked by uid 500); 29 Jan 2014 21:12:12 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 77032 invoked by uid 500); 29 Jan 2014 21:12:10 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 77000 invoked by uid 99); 29 Jan 2014 21:12:09 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Jan 2014 21:12:09 +0000 Date: Wed, 29 Jan 2014 21:12:09 +0000 (UTC) From: "Sushanth Sowmyan (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-6268) Network resource leak with HiveClientCache when using HCatInputFormat MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6268: ----------------------------------- Attachment: HIVE-6268.3.patch One more modification to try and have the first interval where it checks whether to expire from cache or not be early if the user has set a short timeout. > Network resource leak with HiveClientCache when using HCatInputFormat > --------------------------------------------------------------------- > > Key: HIVE-6268 > URL: https://issues.apache.org/jira/browse/HIVE-6268 > Project: Hive > Issue Type: Bug > Components: HCatalog > Affects Versions: 0.12.0 > Reporter: Sushanth Sowmyan > Assignee: Sushanth Sowmyan > Attachments: HIVE-6268.2.patch, HIVE-6268.3.patch, HIVE-6268.patch > > > HCatInputFormat has a cache feature that allows HCat to cache hive client connections to the metastore, so as to not keep reinstantiating a new hive server every single time. This uses a guava cache of hive clients, which only evicts entries from cache on the next write, or by manually managing the cache. > So, in a single threaded case, where we reuse the hive client, the cache works well, but in a massively multithreaded case, where each thread might perform one action, and then is never used, there are no more writes to the cache, and all the clients stay alive, thus keeping ports open. -- This message was sent by Atlassian JIRA (v6.1.5#6160)