hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14881) integrate MM tables into ACID: merge cleaner into ACID threads
Date Thu, 20 Oct 2016 23:16:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593329#comment-15593329
] 

Sergey Shelukhin commented on HIVE-14881:
-----------------------------------------

This will also need to account for list bucketing (current ACID cleaner probably doesn't because
LB doesn't work with bucketing, and thus ACID).
List bucketing looks like partition directories inside partition directories, for a subset
of keys.
MM currently puts MM dirs inside of LB dirs. (e.g. mytable/pkey1=535/pkey2=foo/lbkey=555/mm_0)

If it looks inside delta directories for MM (for whatever reason), it should also be aware
of there being subdirectories there in some cases (at least, union inserts produce subdirectories
for each side of the union; overall, it seems to be legal in Hive to have arbitrary directory
structure inside a table/partition).

> integrate MM tables into ACID: merge cleaner into ACID threads 
> ---------------------------------------------------------------
>
>                 Key: HIVE-14881
>                 URL: https://issues.apache.org/jira/browse/HIVE-14881
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message