hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <>
Subject [jira] [Updated] (HIVE-8258) Compactor cleaners can be starved on a busy table or partition.
Date Thu, 02 Oct 2014 18:00:34 GMT


Alan Gates updated HIVE-8258:
    Status: Patch Available  (was: Open)

In patch 2 changed logic in AcidUtils.getAcidState to only recurse into original directories
once it knows there is no base.  This prevents IOEs from happening if the cleaner is removing
old originals while other readers are reading the new base and delta files.

> Compactor cleaners can be starved on a busy table or partition.
> ---------------------------------------------------------------
>                 Key: HIVE-8258
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.13.1
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>            Priority: Critical
>         Attachments: HIVE-8258.2.patch, HIVE-8258.patch
> Currently the cleaning thread in the compactor does not run on a table or partition while
any locks are held on this partition.  This leaves it open to starvation in the case of a
busy table or partition.  It only needs to wait until all locks on the table/partition at
the time of the compaction have expired.  Any jobs initiated after that (and thus any locks
obtained) will be for the new versions of the files.

This message was sent by Atlassian JIRA

View raw message