aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxim Khutornenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AURORA-1321) Create bulk version of the saveJobInstanceUpdateEvent db store method
Date Fri, 15 May 2015 23:59:59 GMT

    [ https://issues.apache.org/jira/browse/AURORA-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546407#comment-14546407
] 

Maxim Khutornenko commented on AURORA-1321:
-------------------------------------------

I was unable to repro this problem on master. In fact, the batch version of the api behaved
worse than the original version due to dynamic SQL. Digging further, I figured this change
obliterated the perf problem completely: https://reviews.apache.org/r/34046/. Specifically,
setting the "MV_STORE=false" turned off the H2 MVCC, which is infamous for its non-linear
insert performance: http://h2-database.66688.n3.nabble.com/Non-linear-insert-times-in-table-with-FK-td4024729.html.

With the MVCC off, inserting 200K instance events takes only about 4.5 seconds.



> Create bulk version of the saveJobInstanceUpdateEvent db store method
> ---------------------------------------------------------------------
>
>                 Key: AURORA-1321
>                 URL: https://issues.apache.org/jira/browse/AURORA-1321
>             Project: Aurora
>          Issue Type: Bug
>          Components: Scheduler
>            Reporter: Maxim Khutornenko
>            Assignee: Maxim Khutornenko
>
> Perf analysis revealed unsatisfactory performance of the job update instance insertion
implementation during snapshot recovery. It may take up to 5 minutes to process a total of
~100K instance events during scheduler restart. 
> Implement and benchmark the bulk insert version of this api: https://github.com/apache/aurora/blob/b436ec52101cfd5d6f65075a5f7a857ed5a41b5f/src/main/java/org/apache/aurora/scheduler/storage/JobUpdateStore.java#L152



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message