accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <>
Subject [jira] [Updated] (ACCUMULO-1859) Conditional Mutation with 1000 conditions is slow.
Date Thu, 07 Nov 2013 21:51:17 GMT


Keith Turner updated ACCUMULO-1859:

    Attachment: 0002-ACCUMULO-1859-experimental.patch

I ran this bug to the ground and found that DefaultConfiguration was the culprit.   Its iterator()
method could take up to a few milliseconds.  A secondary issue was that each level of the
configuration stack was resorting data (each level was taking everything from the parent and
putting it into a treemap).  Also, unwanted properties were being sorted.   My second patch
addresses all of the issues.   Running the test with the second patch has the same run times
as the first patch.  tl;dr The first patch covered up the performance problem w/ caching,
the second patch fixed it.

> Conditional Mutation with 1000 conditions is slow.
> --------------------------------------------------
>                 Key: ACCUMULO-1859
>                 URL:
>             Project: Accumulo
>          Issue Type: Sub-task
>          Components: client, tserver
>            Reporter: Keith Turner
>            Assignee: Keith Turner
>            Priority: Critical
>             Fix For: 1.6.0
>         Attachments: 0001-ACCUMULO-1859-experimental.patch, 0002-ACCUMULO-1859-experimental.patch
> The random walk test for conditional mutations creates a conditional mutation  with 1000
conditions.  I noticed this was really slow.   
> I did some further testing using MiniAccumulo and was seeing times around 1300ms to 1400ms
for this operation (the conditions all checked for absence and all data for the tablet was
in memory).
> I traced the problem down to parsing the iterator config.  Iterating over the table config,
filtering non iterator props, and parsing was taking a little more than a 1ms.  I modifed
TableConfiguration to cache the iterator props and saw my test times drop to 120ms to 130ms.

This message was sent by Atlassian JIRA

View raw message