accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ara Ebrahimi <ara.ebrah...@argyledata.com>
Subject slowdown in ingest performance
Date Tue, 10 Jan 2017 21:25:55 GMT
Hi,

I’m trying to tune an accumulo 1.7 cluster. We have 3 tablet servers. tservers are configured
to use 32GB of heap (each machine has 256GB mem and 4 disks). These are the configuration
changes we’ve made:

config -s table.cache.block.enable=true
config -s tserver.cache.data.size=4G

config -s tserver.compaction.minor.concurrent.max=50
config -s tserver.compaction.major.concurrent.max=8
config -s tserver.recovery.concurrent.max=4

config -s tserver.wal.replication=2
config -s tserver.wal.blocksize=2G
config -s tserver.walog.max.size=2G
config -s tserver.mutation.queue.max=4M
config -s tserver.memory.maps.max=20G

config -s table.compaction.major.ratio=10
config -s table.compaction.minor.logs.threshold=15
config -s table.file.max=30

config -s table.split.threshold=1024G

config -s table.durability=flush

Reasonable? Any suggestions?

The problem is we see a gradual ingest slow down (see attached screenshot). Why the slowdown?
And how to prevent it? The ingest process is one backed by Kafka and is super fast.

Thanks,
Ara.

[cid:2CCF63FA-4C75-42E6-872F-CF051EF42983]



________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Thank you in advance for your cooperation.

________________________________
Mime
View raw message