ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Pereslegin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-11584) Implement batch insertion of new cache entries in FreeList to improve rebalancing
Date Thu, 01 Aug 2019 09:42:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-11584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865098#comment-16865098
] 

Pavel Pereslegin edited comment on IGNITE-11584 at 8/1/19 9:41 AM:
-------------------------------------------------------------------

Results of microbenchmarks.
Environment: Linux, 16Gb Ram, Core i7 8700
Config: memory page size - 4096 bytes.

1.{{CacheFreeList.insertDataRow()}} vs {{CacheFreeList.insertDataRows()}}.
Source code: [https://gist.github.com/xtern/8443638a026785655f1cd4d084ded6fd]

Benchmark measures the time required to insert 100 data rows of different sizes (the size
does not include object overhead ~40 bytes).
||size (bytes)|*4 - 64*|*100-300*|*300-700*|*700 - 1200*|*1200 - 3000*|             |*1000
- 8000*|*4000 - 16000*|*100 - 32000*|
||single (μs)|162.3 ^± 4.2^|140.7  ^± 1.2^|159.9  ^± 4.6^|175.2  ^± 1.6^|239.8 
^± 2.3^| |422.3  ^± 5.7^|867.9  ^± 89.3^|1287.0  ^± 55.8^|
||batch (μs)|28.0  ^± 0.9^|43.4  ^± 0.4^|74.6  ^± 0.7^|115.8  ^± 2.0^|232.9  ^±
5.7^| |398.5  ^± 8.6^|794.6  ^± 8.7^|1209.0  ^± 20.9^|

2. Comparison of preloading performance (master branch vs patch branch).
Source code: [https://gist.github.com/xtern/4a4699efd06f147df2b7b342169aee0d]

Benchmark measures the time required for handling one supply message with 100 objects of different
sizes (the size does not include object overhead ~40 bytes).
||size (bytes)|*4 - 64*|*100-300*|*300-700*|*700 - 1200*|*1200 - 3000*|             |*1000
- 8000*|*4000 - 16000*| *100 - 32000*|
||master(μs)|198.7  ^± 7.2^|205.7  ^± 10.0^|213.3  ^± 17.4^|243.4  ^± 31.7^|261.2 
^± 10.6^| |371.8  ^± 13.5^|639.5  ^± 36.4^|914.5  ^± 85.5^|
||patch (μs)|121.9  ^± 4.1^|141.3  ^± 23.0^|155.3  ^± 15.8^|178.7  ^± 21.4^|241.2 
^± 6.3^| |359.1  ^± 31.3^|637.3  ^± 145.7^|898.6  ^± 81.6^|

Free list benchmark shows that performance increases in cases when the memory page has enough
free space to store more than one object.
The performance boost in preloading is significantly lower since this process involves more
overhead. Real life rebalancing speedup will be even less.


was (Author: xtern):
Results of microbenchmarks.
Environment: Linux, 16Gb Ram, Core i7 8700
Config: memory page size - 4096 bytes.

1.{{CacheFreeList.insertDataRow()}} vs {{CacheFreeList.insertDataRows()}}.
Source code: [https://gist.github.com/xtern/8443638a026785655f1cd4d084ded6fd
]Benchmark measures the time required to insert 100 data rows of different sizes (the size
does not include object overhead ~40 bytes).
||size (bytes)|*4 - 64*|*100-300*|*300-700*|*700 - 1200*|*1200 - 3000*|             |*1000
- 8000*|*4000 - 16000*|*100 - 32000*|
||single (μs)|162.3 ^± 4.2^|140.7  ^± 1.2^|159.9  ^± 4.6^|175.2  ^± 1.6^|239.8 
^± 2.3^| |422.3  ^± 5.7^|867.9  ^± 89.3^|1287.0  ^± 55.8^|
||batch (μs)|28.0  ^± 0.9^|43.4  ^± 0.4^|74.6  ^± 0.7^|115.8  ^± 2.0^|232.9  ^±
5.7^| |398.5  ^± 8.6^|794.6  ^± 8.7^|1209.0  ^± 20.9^|

2. Comparison of preloading performance (master branch vs patch branch).
Source code: [https://gist.github.com/xtern/4a4699efd06f147df2b7b342169aee0d
]Benchmark measures the time required for handling one supply message with 100 objects of
different sizes (the size does not include object overhead ~40 bytes).
||size (bytes)|*4 - 64*|*100-300*|*300-700*|*700 - 1200*|*1200 - 3000*|             |*1000
- 8000*|*4000 - 16000*| *100 - 32000*|
||master(μs)|198.7  ^± 7.2^|205.7  ^± 10.0^|213.3  ^± 17.4^|243.4  ^± 31.7^|261.2 
^± 10.6^| |371.8  ^± 13.5^|639.5  ^± 36.4^|914.5  ^± 85.5^|
||patch (μs)|121.9  ^± 4.1^|141.3  ^± 23.0^|155.3  ^± 15.8^|178.7  ^± 21.4^|241.2 
^± 6.3^| |359.1  ^± 31.3^|637.3  ^± 145.7^|898.6  ^± 81.6^|

Free list benchmark shows that performance increases in cases when the memory page has enough
free space to store more than one object.
The performance boost in preloading is significantly lower since this process involves more
overhead. Real life rebalancing speedup will be even less.

> Implement batch insertion of new cache entries in FreeList to improve rebalancing
> ---------------------------------------------------------------------------------
>
>                 Key: IGNITE-11584
>                 URL: https://issues.apache.org/jira/browse/IGNITE-11584
>             Project: Ignite
>          Issue Type: Sub-task
>    Affects Versions: 2.7
>            Reporter: Pavel Pereslegin
>            Assignee: Pavel Pereslegin
>            Priority: Major
>              Labels: iep-32
>             Fix For: 2.8
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Main goals:
>  * Implement batch insert operation into FreeList - insert several data rows at once
>  * Use batch insertion in the preloader
>   
> Implementation notes:
>  # Preloader cannot lock multiple cache entries at once, because this may lead to a deadlock
with concurrent batch updates. Therefore, it pre-creates batch of data rows in the page memory,
and then sequentially initializes the cache entries one by one.
>  # Batch writing of data rows into data pages uses the free list as usual because other
approaches increase memory fragmentation (for example, using only "reuse" or "most free" buckets).
>  # Eviction tracker assumes that only data pages with "heads" of fragmented data row
are tracked, so all other fragments of large data row should be written on separate data pages
(without other data rows which may cause page tracking).



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message