Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 26 Nov 2014 22:19:13 +0000 (UTC)
From: "Alan Gates (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12757795.1416947573000.30345.1417040353269@Atlassian.JIRA>
In-Reply-To: <JIRA.12757795.1416947573000@Atlassian.JIRA>
References: <JIRA.12757795.1416947573000@Atlassian.JIRA>
 <JIRA.12757795.1416947573814@arcas>
Subject: [jira] [Commented] (HIVE-8966) Delta files created by hive hcatalog
 streaming cannot be compacted
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226890#comment-14226890 ] 

Alan Gates commented on HIVE-8966:
----------------------------------

1 might be the right thing to do.  2 breaks backward compatibility.  Before we do that though I'd like to understand why you still see the flush length files hanging around.  In my tests I don't see this issue because the flush length file is properly cleaned up.  I want to make sure that its existence doesn't mean something else is wrong.

Do you see the flush length files in all delta directories or only the most recent?  

> Delta files created by hive hcatalog streaming cannot be compacted
> ------------------------------------------------------------------
>
>                 Key: HIVE-8966
>                 URL: https://issues.apache.org/jira/browse/HIVE-8966
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 0.14.0
>         Environment: hive
>            Reporter: Jihong Liu
>            Assignee: Alan Gates
>            Priority: Critical
>
> hive hcatalog streaming will also create a file like bucket_n_flush_length in each delta directory. Where "n" is the bucket number. But the compactor.CompactorMR think this file also needs to compact. However this file of course cannot be compacted, so compactor.CompactorMR will not continue to do the compaction. 
> Did a test, after removed the bucket_n_flush_length file, then the "alter table partition compact" finished successfully. If don't delete that file, nothing will be compacted. 
> This is probably a very severity bug. Both 0.13 and 0.14 have this issue


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)