hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <>
Subject [jira] [Commented] (HIVE-12353) When Compactor fails it calls CompactionTxnHandler.markedCleaned(). it should not.
Date Thu, 21 Jan 2016 09:23:39 GMT


Hive QA commented on HIVE-12353:

Here are the results of testing the latest attachment:

{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10010 tests executed
*Failed tests:*
TestHWISessionManager - did not produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more -
did not produce a TEST-*.xml file

Test results:
Console output:
Test logs:

Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed

This message is automatically generated.

ATTACHMENT ID: 12783461 - PreCommit-HIVE-TRUNK-Build

> When Compactor fails it calls CompactionTxnHandler.markedCleaned().  it should not.
> -----------------------------------------------------------------------------------
>                 Key: HIVE-12353
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Blocker
>         Attachments: HIVE-12353.2.patch, HIVE-12353.3.patch, HIVE-12353.4.patch, HIVE-12353.6.patch,
HIVE-12353.7.patch, HIVE-12353.patch
> One of the things that this method does is delete entries from TXN_COMPONENTS for partition
that it was trying to compact.
> This causes Aborted transactions in TXNS to become empty according to
> CompactionTxnHandler.cleanEmptyAbortedTxns() which means they can now be deleted.  
> Once they are deleted, data that belongs to these txns is deemed committed...
> We should extend COMPACTION_QUEUE state with 'f' and 's' (failed, success) states.  We
should also not delete then entry from markedCleaned()
> We'll have separate process that cleans 'f' and 's' records after X minutes (or after
> N records for a given partition exist).
> This allows SHOW COMPACTIONS to show some history info and how many times compaction
failed on a given partition (subject to retention interval) so that we don't have to call
markCleaned() on Compactor failures at the same time preventing Compactor to constantly getting
stuck on the same bad partition/table.
> Ideally we'd want to include END_TIME field.

This message was sent by Atlassian JIRA

View raw message