accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Kepner (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-931) Oscillations in Accumulo Ingest Performance
Date Mon, 07 Jan 2013 15:12:12 GMT


Jeremy Kepner commented on ACCUMULO-931:

So the phenomena appears to be that  mutations can return control to the ingestor before the
minor compactions they cause have completed.  Thus a single ingestor can cause multiple simultaneous
compactions on the same tablet.  If the table only has one tablet this cause the ingest process
back up (see file 1ingestor_1table_1tablet.pdf).  The solution is to presplit the tablet.
 The files 1ingestor_1table_2tablet.pdf and 1ingestor_1table_4tablet.pdf shows that these
splits solves the problem.  The file 2,3,4ingestor_1table_4tablet.pdf show the performance
of three separate runs using 2, 3, and 4 ingestors into a table with 4 tablets..  Likewise,
the file 10,r8,r10,r12ingestor_1table_12tablet.pdf  shows four separate runs using 10 local,
8 remote, 10 remote, and 12 remote ingestors.  In all cases, the splitting resolves the performance

If there are no objections, this issue can be closed.
> Oscillations in Accumulo Ingest Performance
> -------------------------------------------
>                 Key: ACCUMULO-931
>                 URL:
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.2
>         Environment: 
>            Reporter: Jeremy Kepner
>         Attachments: 1ingestor_1table_1tablet.pdf, 1ingestor.pdf, 2ingestor.pdf, 3ingestor.pdf,
4ingestor.pdf, ingest_performance_explained.pdf
> Linux 2.6.32, single node, 32 cores, 96 GB RAM, 3x3TB SATA drives, RAID5
> Ingest performance into accumulo varies by 2.5x.  Depending upon the number of ingestors.
> Performance tests were carried out using Graph500 benchmark (see d4m_api/examples/3Scaling/2ParallelDatabase/pDB10_EdgeInsertTEST.m

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message