Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E5489200C29 for ; Tue, 28 Feb 2017 20:59:52 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E3D84160B7C; Tue, 28 Feb 2017 19:59:52 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3A34C160B59 for ; Tue, 28 Feb 2017 20:59:52 +0100 (CET) Received: (qmail 11895 invoked by uid 500); 28 Feb 2017 19:59:51 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 11884 invoked by uid 99); 28 Feb 2017 19:59:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Feb 2017 19:59:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C9E63C094B for ; Tue, 28 Feb 2017 19:59:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -2.346 X-Spam-Level: X-Spam-Status: No, score=-2.346 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-2.999, SPF_NEUTRAL=0.652, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 9TspuMgzwGya for ; Tue, 28 Feb 2017 19:59:50 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 82CD05F476 for ; Tue, 28 Feb 2017 19:59:49 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 046F5E0629 for ; Tue, 28 Feb 2017 19:59:46 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 6647624160 for ; Tue, 28 Feb 2017 19:59:45 +0000 (UTC) Date: Tue, 28 Feb 2017 19:59:45 +0000 (UTC) From: "Chao Sun (JIRA)" To: dev@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HIVE-16060) GenericUDTFJSONTuple's HashCache could overgrown MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 28 Feb 2017 19:59:53 -0000 Chao Sun created HIVE-16060: ------------------------------- Summary: GenericUDTFJSONTuple's HashCache could overgrown Key: HIVE-16060 URL: https://issues.apache.org/jira/browse/HIVE-16060 Project: Hive Issue Type: Bug Components: UDF Reporter: Chao Sun Assignee: Chao Sun At the moment the [cache object|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFJSONTuple.java#L96] used in {{GenericUDTFJSONTuple}} is a static linked hashmap that is not thread-safe. In the case of HoS it may be accessed concurrently and has race conditions. In particular, its size may overgrow even though the limit is 32. This can be observed from the attached image. An easy way to fix it is to make it non-static. -- This message was sent by Atlassian JIRA (v6.3.15#6346)