From dev-return-54206-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Sat Oct 6 10:27:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 62DA218061A for ; Sat, 6 Oct 2018 10:27:05 +0200 (CEST) Received: (qmail 89954 invoked by uid 500); 6 Oct 2018 08:27:03 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 89943 invoked by uid 99); 6 Oct 2018 08:27:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Oct 2018 08:27:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id CC4C61A0EFF for ; Sat, 6 Oct 2018 08:27:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id WFu8p6xc5-uM for ; Sat, 6 Oct 2018 08:27:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 79DFC5F217 for ; Sat, 6 Oct 2018 08:27:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 96056E00A5 for ; Sat, 6 Oct 2018 08:27:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 100D424727 for ; Sat, 6 Oct 2018 08:27:00 +0000 (UTC) Date: Sat, 6 Oct 2018 08:27:00 +0000 (UTC) From: "Volodymyr Kvych (JIRA)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (PHOENIX-4958) Hbase does not load updated UDF class simultaneously on whole cluster MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Volodymyr Kvych created PHOENIX-4958: ---------------------------------------- Summary: Hbase does not load updated UDF class simultaneously on whole cluster Key: PHOENIX-4958 URL: https://issues.apache.org/jira/browse/PHOENIX-4958 Project: Phoenix Issue Type: Bug Reporter: Volodymyr Kvych To update UDF according to [https://phoenix.apache.org/udf.html] limitations, I do next steps: # Drop existing function and JAR file: {code:sql} DROP FUNCTION my_function; DELETE JAR 'hdfs:/.../udf-v1.jar;{code} # Remove JAR file across cluster's local file system, like: {code:java} rm ${hbase.local.dir}/jars/udf-v1.jar{code} # Upload updated JAR file and create the same function: {code:sql} ADD JARS '/.../udf-v2.jar; CREATE FUNCTION my_function(...) ... using jar 'hdfs:/.../udf-v2.jar'; {code} The problem is, that every RegionServer could keep the previously loaded function undefined period of time until GC decides to collect appropriate DynamicClassLoader instance and old UDF class. As result, some RegionServer might execute new function's code, but others - the old one. There is no way to ensure that the function was reloaded by whole cluster. As a proposed fix, I'd updated the UDFExpression to keep DynamicClassLoaders per-tenant and per-jar name. Since JAR name must be changed to correctly update the UDF, it's working for described use case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)