ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "BinaryTree" <binarytre...@foxmail.com>
Subject Backup make DataStreamer performance decreased a lot.
Date Fri, 01 Mar 2019 09:53:11 GMT
Hi Igniters - 

I know backups will impact the performance of the cluster:

If you use a PARTITIONED cache and the data loss is not critical for you (for example, when
you have a backing cache store), consider disabling backups for the cache. When backups are
enabled, the cache engine has to maintain a remote copy of each entry, which requires network
exchange and is time-consuming.

Because the data is important and can not lose, so the backup is necessary.

But the backup make DataStreamer performance decreased a lot, if backups are disabled,  40
million records can be loaded in 4 minutes, but when set backup  = 1, after loading 20 million
records, the speed decreased a lot, sometimes, it will cost more 20 seconds to load 10 thousands
records.

Are there any configurations or methods can improve the performace of DataStreamer?

Related post:

http://apache-ignite-users.70518.x6.nabble.com/Ignite-Data-Streamer-Hung-after-a-period-tp21161.html

I attached the thread dumps in this post.

I also create a project to reproduce the problem, you can refer to :

https://github.com/RedBlackTreei/streamer.git
Mime
View raw message