accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Kepner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-931) Oscillations in Accumulo Ingest Performance
Date Sat, 05 Jan 2013 21:14:13 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13545173#comment-13545173
] 

Jeremy Kepner commented on ACCUMULO-931:
----------------------------------------

Based on feedback I have simplified the experiment to 1 ingestor, 1 table, and 1 tablet. 
In this simpler experiment it is clear that the drop in ingest performance is correlated with
the increase in the number of minor compactions.  The table reports that it is on only 1 tablet.
 If this is to be believed, then multiple tablets isn't the source of multiple compactions.
 If it is the case that the only other way that multiple compactions can be occurring is if
there are multiple ingestors, then that is what must be happening.  The benchmark starts a
new mutation as soon as control is returned to the main program.  If it is possible for control
to be returned prior to the mutation completing then this would be an explanation for how
multiple minor compactions could be taking place.
                
> Oscillations in Accumulo Ingest Performance
> -------------------------------------------
>
>                 Key: ACCUMULO-931
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-931
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.2
>         Environment: 
>            Reporter: Jeremy Kepner
>         Attachments: 1ingestor_1table_1tablet.pdf, 1ingestor.pdf, 2ingestor.pdf, 3ingestor.pdf,
4ingestor.pdf, ingest_performance_explained.pdf
>
>
> Linux 2.6.32, single node, 32 cores, 96 GB RAM, 3x3TB SATA drives, RAID5
> Ingest performance into accumulo varies by 2.5x.  Depending upon the number of ingestors.
> Performance tests were carried out using Graph500 benchmark (see d4m_api/examples/3Scaling/2ParallelDatabase/pDB10_EdgeInsertTEST.m
from http://www.mit.edu/~kepner/D4M/).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message