Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 22 Dec 2016 20:05:58 +0000 (UTC)
From: "Dongjoon Hyun (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.13029402.1482227963000.584349.1482437158430@Atlassian.JIRA>
In-Reply-To: <JIRA.13029402.1482227963000@Atlassian.JIRA>
References: <JIRA.13029402.1482227963000@Atlassian.JIRA> <JIRA.13029402.1482227963155@arcas>
Subject: [jira] [Comment Edited] (SPARK-18941) Spark thrift server, Spark
 2.0.2, The "drop table" command doesn't delete the directory associated
 with the Hive table (not EXTERNAL table) from the HDFS file system
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 22 Dec 2016 20:06:00 -0000


    [ https://issues.apache.org/jira/browse/SPARK-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15770967#comment-15770967 ] 

Dongjoon Hyun edited comment on SPARK-18941 at 12/22/16 8:05 PM:
-----------------------------------------------------------------

Hi, first of all. your case is correct. I can reproduce your example. Thanks.

Currently, it seems to be intentional behavior because Spark assumes your table is EXTERNAL when users give locations.

{code}
scala> sql("create table table_with_location(a int) stored as orc location '/tmp/table_with_location'")
scala> sql("desc extended table_with_location").show(false)
...
|# Detailed Table Information|CatalogTable(
	Table: `default`.`table_with_location`
	Owner: dhyun
	Created: Thu Dec 22 12:01:35 PST 2016
	Last Access: Wed Dec 31 16:00:00 PST 1969
	Type: EXTERNAL
	Schema: [StructField(a,IntegerType,true)]
	Provider: hive
	Properties: [transient_lastDdlTime=1482436895]
...
{code}

Let me try to make a PR for this.


was (Author: dongjoon):
Hi, first of all. your case is correct. I can reproduce your example. Thanks.

Currently, it seems to be intentional behavior because Spark assumes your table is EXTERNAL when users give locations.

```
scala> sql("create table table_with_location(a int) stored as orc location '/tmp/table_with_location'")
res0: org.apache.spark.sql.DataFrame = []

scala> sql("desc extended table_with_location").show(false)
...
|# Detailed Table Information|CatalogTable(
	Table: `default`.`table_with_location`
	Owner: dhyun
	Created: Thu Dec 22 12:01:35 PST 2016
	Last Access: Wed Dec 31 16:00:00 PST 1969
	Type: EXTERNAL
	Schema: [StructField(a,IntegerType,true)]
	Provider: hive
	Properties: [transient_lastDdlTime=1482436895]
...
```

Let me try to make a PR for this.

> Spark thrift server, Spark 2.0.2, The "drop table" command doesn't delete the directory associated with the Hive table (not EXTERNAL table) from the HDFS file system
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-18941
>                 URL: https://issues.apache.org/jira/browse/SPARK-18941
>             Project: Spark
>          Issue Type: Bug
>          Components: Java API
>    Affects Versions: 2.0.2
>            Reporter: luat
>
> Spark thrift server, Spark 2.0.2, The "drop table" command doesn't delete the directory associated with the Hive table (not EXTERNAL table) from the HDFS file system.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org