Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0CEC710FE4 for ; Sat, 13 Dec 2014 00:11:14 +0000 (UTC) Received: (qmail 70851 invoked by uid 500); 13 Dec 2014 00:11:13 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 70780 invoked by uid 500); 13 Dec 2014 00:11:13 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 70539 invoked by uid 500); 13 Dec 2014 00:11:13 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 70499 invoked by uid 99); 13 Dec 2014 00:11:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Dec 2014 00:11:13 +0000 Date: Sat, 13 Dec 2014 00:11:13 +0000 (UTC) From: "Xuefu Zhang (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch] MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245020#comment-14245020 ] Xuefu Zhang commented on HIVE-9017: ----------------------------------- To clarify, when Spark lunched multiple executors in one host for one application, these executors share the same JVM, right? At least that's my understanding. On the same host, there may be other JVMs, but they will be for different applications. Different JVMs, and thus different applications, shouldn't share the cache libs or data. That's my understanding, but I could be bogus on this. I can understand that Spark doesn't want each executor of an application to download the same files. All executors in one JVM can share one copy of the files, as these executors are for one application only. That's what I think SPARK-2713 is for. > Clean up temp files of RSC [Spark Branch] > ----------------------------------------- > > Key: HIVE-9017 > URL: https://issues.apache.org/jira/browse/HIVE-9017 > Project: Hive > Issue Type: Sub-task > Components: Spark > Reporter: Rui Li > > Currently RSC will leave a lot of temp files in {{/tmp}}, including {{*_lock}}, {{*_cache}}, {{spark-submit.*.properties}}, etc. > We should clean up these files or it will exhaust disk space. -- This message was sent by Atlassian JIRA (v6.3.4#6332)