spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srowen <>
Subject [GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...
Date Mon, 24 Sep 2018 14:58:18 GMT
Github user srowen commented on the issue:
    There is ... see
    I think Spark conflates the two. It's rare (?) but possible to specify a custom location
of a managed table, but, typically occurs for `EXTERNAL` tables. So maybe this is OK.
      private def createTable(tableIdent: TableIdentifier): Unit = {
        val storage = DataSource.buildStorageFormatFromOptions(extraOptions.toMap)
        val tableType = if (storage.locationUri.isDefined) {
        } else {
    And in `SqlParser`:
        // If location is defined, we'll assume this is an external table.
        // Otherwise, we may accidentally delete existing data.
        val tableType = if (external || location.isDefined) {
        } else {
    So if `LOCATION` implies `EXTERNAL` in Spark, then I get this. `EXTERNAL` tables shouldn't
be deleted.
    I agree that the Hive impl doesn't seem to take this into account, on the code paths that
call `dropDatabase`. CC @andrewor14 in case he is available to comment on the original implementaiton.
WDYT @cloud-fan 


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message