From dev-return-150468-archive-asf-public=cust-asf.ponee.io@hive.apache.org Mon Oct 15 18:54:35 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5C138180647 for ; Mon, 15 Oct 2018 18:54:35 +0200 (CEST) Received: (qmail 4106 invoked by uid 500); 15 Oct 2018 16:54:34 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 4094 invoked by uid 99); 15 Oct 2018 16:54:33 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Oct 2018 16:54:33 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 38ED71A073C; Mon, 15 Oct 2018 16:54:33 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id BoGBNX0pxPNU; Mon, 15 Oct 2018 16:54:31 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id EBDE85F43C; Mon, 15 Oct 2018 16:54:30 +0000 (UTC) Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id A50F4E0177; Mon, 15 Oct 2018 16:54:30 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id 5896FC402E0; Mon, 15 Oct 2018 16:54:30 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============7051759788295733176==" MIME-Version: 1.0 Subject: Re: Review Request 69022: HIVE-20737: SparkContext is shared between user sessions and should be closed only when there is no active one From: denys kuzmenko via Review Board To: Sahil Takiar , Adam Szita Cc: denys kuzmenko , hive Date: Mon, 15 Oct 2018 16:54:30 -0000 Message-ID: <20181015165430.33136.92853@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: denys kuzmenko X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/69022/ X-Sender: denys kuzmenko References: <20181015095734.33136.79702@reviews-vm2.apache.org> In-Reply-To: <20181015095734.33136.79702@reviews-vm2.apache.org> X-ReviewBoard-Diff-For: ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestLocalHiveSparkClient.java Reply-To: denys kuzmenko X-ReviewRequest-Repository: hive-git --===============7051759788295733176== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69022/ ----------------------------------------------------------- (Updated Oct. 15, 2018, 4:54 p.m.) Review request for hive, Sahil Takiar and Adam Szita. Summary (updated) ----------------- HIVE-20737: SparkContext is shared between user sessions and should be closed only when there is no active one Bugs: HIVE-20737 https://issues.apache.org/jira/browse/HIVE-20737 Repository: hive-git Description (updated) ------- 1. SparkContext is shared between user sessions and should be closed only when there is no active one. 2. Possible race condition in SparkSession.open() in case of user queries run in parallel within a single session. Diffs (updated) ----- ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java 72ff53e3bd ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java bb50129518 ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestLocalHiveSparkClient.java PRE-CREATION Diff: https://reviews.apache.org/r/69022/diff/2/ Changes: https://reviews.apache.org/r/69022/diff/1-2/ Testing ------- Added TestLocalHiveSparkClient test File Attachments (updated) ---------------- HIVE-20737.7.patch https://reviews.apache.org/media/uploaded/files/2018/10/15/9cf8a2b3-9ec1-4316-81d0-3cd124b1a9fd__HIVE-20737.7.patch Thanks, denys kuzmenko --===============7051759788295733176==--