accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3974) Modify Randomwalk Bulk module to catch ACCUMULO-3967
Date Thu, 27 Aug 2015 15:34:46 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716866#comment-14716866
] 

Josh Elser commented on ACCUMULO-3974:
--------------------------------------

bq. Another option to consider is refactoring some of the bulk import code to make it more
testable

That's a good idea as well. I didn't do this in the older branches for fear of breaking something
else. Maybe adequate unit testing of BulkImporter is sufficient to not need a high-level test
for it?

Either way, more testing here is good, but I'm getting the impression that everyone is happy
with the level of testing already in place for 1.5.4, 1.6.4 and 1.7.1.

> Modify Randomwalk Bulk module to catch ACCUMULO-3967
> ----------------------------------------------------
>
>                 Key: ACCUMULO-3974
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3974
>             Project: Accumulo
>          Issue Type: Bug
>          Components: test
>            Reporter: Josh Elser
>            Priority: Critical
>             Fix For: 1.6.4, 1.7.1, 1.8.0
>
>
> [~ecn] asked me after I committed a fix for ACCUMULO-3967 "why didn't randomwalk catch
this bug?"
> I think we could potentially have caught this if in Setup we pre-split the table to be
a large collection of sequentially increasing tablets. It's not a guarantee catch (since the
bug itself was only shown in the case of import failures and the tablet distribution hasn't
changed).
> Alternatively, we could copy the existing bulk module into a new module. In this module,
we remove the splitting and merging, instead keeping a static split distribution. This would
be almost guaranteed to eventually recreate the scenario. Adding in a chaotic balancer, even
more likely to reproduce.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message