accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Kepner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-931) Oscillations in Accumulo Ingest Performance
Date Mon, 07 Jan 2013 15:12:12 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13545949#comment-13545949
] 

Jeremy Kepner commented on ACCUMULO-931:
----------------------------------------

So the phenomena appears to be that  mutations can return control to the ingestor before the
minor compactions they cause have completed.  Thus a single ingestor can cause multiple simultaneous
compactions on the same tablet.  If the table only has one tablet this cause the ingest process
back up (see file 1ingestor_1table_1tablet.pdf).  The solution is to presplit the tablet.
 The files 1ingestor_1table_2tablet.pdf and 1ingestor_1table_4tablet.pdf shows that these
splits solves the problem.  The file 2,3,4ingestor_1table_4tablet.pdf show the performance
of three separate runs using 2, 3, and 4 ingestors into a table with 4 tablets..  Likewise,
the file 10,r8,r10,r12ingestor_1table_12tablet.pdf  shows four separate runs using 10 local,
8 remote, 10 remote, and 12 remote ingestors.  In all cases, the splitting resolves the performance
issue.

If there are no objections, this issue can be closed.
                
> Oscillations in Accumulo Ingest Performance
> -------------------------------------------
>
>                 Key: ACCUMULO-931
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-931
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.2
>         Environment: 
>            Reporter: Jeremy Kepner
>         Attachments: 1ingestor_1table_1tablet.pdf, 1ingestor.pdf, 2ingestor.pdf, 3ingestor.pdf,
4ingestor.pdf, ingest_performance_explained.pdf
>
>
> Linux 2.6.32, single node, 32 cores, 96 GB RAM, 3x3TB SATA drives, RAID5
> Ingest performance into accumulo varies by 2.5x.  Depending upon the number of ingestors.
> Performance tests were carried out using Graph500 benchmark (see d4m_api/examples/3Scaling/2ParallelDatabase/pDB10_EdgeInsertTEST.m
from http://www.mit.edu/~kepner/D4M/).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message