Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id AA059200BB4 for ; Tue, 1 Nov 2016 21:05:00 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id A8773160AF7; Tue, 1 Nov 2016 20:05:00 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 196E7160ADA for ; Tue, 1 Nov 2016 21:04:59 +0100 (CET) Received: (qmail 9180 invoked by uid 500); 1 Nov 2016 20:04:59 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 9168 invoked by uid 99); 1 Nov 2016 20:04:59 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Nov 2016 20:04:59 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id DA7562C2A6B for ; Tue, 1 Nov 2016 20:04:58 +0000 (UTC) Date: Tue, 1 Nov 2016 20:04:58 +0000 (UTC) From: "Daniel Dai (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-15068) Run ClearDanglingScratchDir periodically inside HS2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 01 Nov 2016 20:05:00 -0000 [ https://issues.apache.org/jira/browse/HIVE-15068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-15068: ------------------------------ Attachment: (was: HIVE-15068.3.patch) > Run ClearDanglingScratchDir periodically inside HS2 > --------------------------------------------------- > > Key: HIVE-15068 > URL: https://issues.apache.org/jira/browse/HIVE-15068 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 > Reporter: Daniel Dai > Assignee: Daniel Dai > Attachments: HIVE-15068.1.patch, HIVE-15068.2.patch > > > In HIVE-13429, we introduce a tool which clear dangling scratch directory. In this ticket, we want to invoke the tool automatically on a Hive cluster. Options are: > 1. cron job, which would involve manual cron job setup > 2. As a metastore thread. However, it is possible we run metastore without hdfs in the future (eg, managing s3 files). ClearDanglingScratchDir needs support which only exists in hdfs, it won't work if the above scenario happens > 3. As a HS2 thread. The downside is if no HS2 is running, the tool will not run automatically. But we expect HS2 will be a required component down the road > Here I choose approach 3 in the implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)