accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (ACCUMULO-1044) bulk imported files showing up in metadata after bulk import fails
Date Tue, 09 Apr 2013 19:34:16 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626990#comment-13626990
] 

Keith Turner edited comment on ACCUMULO-1044 at 4/9/13 7:33 PM:
----------------------------------------------------------------

A workaround for 1.4.3 and earlier is to remove the !METADATA compaction filter that drops
bulk import load flags.

{noformat}
  deleteiter -majc -n bulkLoadFilter -t !METADATA
{noformat}

Then periodically delete bulk load flags that are left by splits.  Would need to add the filter
back once this bug is fixed.

I think a solution to fix the bug is to write two things to zookeeper for a bulk import transactions.
 Delete the first marker in zookeeper when you want work related to the bulk import to stop.
 Delete the 2nd marker when its safe for major compaction to delete bulk load markers.
                
      was (Author: kturner):
    A workaround for 1.4.3 and earlier is to remove the !METADATA compaction filter that drops
bulk import load flags.

{noformat}
  deleteiter -majc -n bulkLoadFilter -t !METADATA
{noformat}

Then periodically delete bulk load flags that are left by splits.  Would need to add the filter
back once this bug is fixed.

I think a solution to fix the bug is to write to things to zookeeper for a bulk import transactions.
 Delete the first marker in zookeeper when you want work related to the bulk import to stop.
 Delete the 2nd marker when its safe for major compaction to delete bulk load markers.
                  
> bulk imported files showing up in metadata after bulk import fails
> ------------------------------------------------------------------
>
>                 Key: ACCUMULO-1044
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1044
>             Project: Accumulo
>          Issue Type: Bug
>          Components: master, tserver
>    Affects Versions: 1.4.2
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>            Priority: Critical
>              Labels: 15_qa_bug
>             Fix For: 1.6.0
>
>
> Bulk import fails.  The file is moved to the failures directory.
> But references in the !METADATA table remain.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message