From dev-return-60543-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Sat Mar 28 01:36:03 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4A6B0180637 for ; Sat, 28 Mar 2020 02:36:03 +0100 (CET) Received: (qmail 70037 invoked by uid 500); 28 Mar 2020 01:36:02 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 69794 invoked by uid 99); 28 Mar 2020 01:36:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 28 Mar 2020 01:36:01 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D6A7EE2E39 for ; Sat, 28 Mar 2020 01:36:00 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 34B46780642 for ; Sat, 28 Mar 2020 01:36:00 +0000 (UTC) Date: Sat, 28 Mar 2020 01:36:00 +0000 (UTC) From: "Chinmay Kulkarni (Jira)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PHOENIX-5802) Connection leaks in UPSERT SELECT/DELETE paths due to MutatingParallelIteratorFactory iterator not being closed MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-5802?page=3Dcom.atlass= ian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated PHOENIX-5802: -------------------------------------- Attachment: PHOENIX-5802-4.x-v2.patch > Connection leaks in UPSERT SELECT/DELETE paths due to MutatingParallelIte= ratorFactory iterator not being closed > -------------------------------------------------------------------------= -------------------------------------- > > Key: PHOENIX-5802 > URL: https://issues.apache.org/jira/browse/PHOENIX-5802 > Project: Phoenix > Issue Type: Bug > Affects Versions: 5.0.0, 4.15.0 > Reporter: Chinmay Kulkarni > Assignee: Chinmay Kulkarni > Priority: Major > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-5802-4.x-v1.patch, PHOENIX-5802-4.x-v2.patch > > Time Spent: 3h > Remaining Estimate: 0h > > During an UPSERT SELECT query or client-side DELETE, we clone the existin= g Phoenix connection and call [mutate|https://github.com/apache/phoenix/blo= b/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/a= pache/phoenix/compile/MutatingParallelIteratorFactory.java#L59] which event= ually calls either [upsertSelect|https://github.com/apache/phoenix/blob/73a= 86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache= /phoenix/compile/UpsertCompiler.java#L178] or [deleteRows|https://github.co= m/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core= /src/main/java/org/apache/phoenix/compile/DeleteCompiler.java#L126]. > When getting iterators, if we encounter any exception, then on calling [c= lose|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de412392= 11330e7a7/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultI= terators.java#L1377], we attempt to cancel any queued work, while [accumula= ting already-started task futures|https://github.com/apache/phoenix/blob/73= a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apach= e/phoenix/iterate/BaseResultIterators.java#L1453].=20 > Later we attempt to block on getting the result (iterator) of already-st= arted tasks and [close each iterator|https://github.com/apache/phoenix/blob= /73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/ap= ache/phoenix/iterate/BaseResultIterators.java#L1464-L1465]. > Here however, if any of the upsert-select or client-side delete tasks had= thrown an exception _E_, we wouldn't be able to close the underlying itera= tor (which in-turn would've=C2=A0[closed the cloned connection|https://gith= ub.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix= -core/src/main/java/org/apache/phoenix/compile/MutatingParallelIteratorFact= ory.java#L101]). This manifests as [these logs|https://github.com/apache/ph= oenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/j= ava/org/apache/phoenix/iterate/BaseResultIterators.java#L1470]: > {code:java} > "Failed to execute task during cancel", > {code} > This is leading to a connection leak. Because we use CQSI to create the P= hoenix Connection here, it also gets accounted for during open Phoenix conn= ection throttling, thus leaving the client application with less connection= s that can be opened. -- This message was sent by Atlassian Jira (v8.3.4#803005)