flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyula.f...@gmail.com>
Subject Re: RocksDB native checkpoint time
Date Tue, 14 May 2019 07:41:12 GMT
Hey,

I have collected some rocksdb logs for the snapshot itself but I cant
really wrap my head around where exactly the time is spent:
https://gist.github.com/gyfora/9a37aa349f63c35cd6abe2da2cf19d5b

The general pattern where the time is spent is this:
2019/05/14-09:15:49.486455 7fbe6a8ee700 [db/db_impl_write.cc:1127]
[new-timer-state] New memtable created with log file: #111757. Immutable
memtables: 0.
2019/05/14-09:15:59.191010 7fb3cdc1d700 (Original Log Time
2019/05/14-09:15:59.191000) [db/db_impl_compaction_flush.cc:1216] Calling
FlushMemTableToOutputFile with column family [new-timer-state], flush slots
available 1, compaction slots available 1, flush slots scheduled 1,
compaction slots scheduled 0

In this example these two operations take 10 seconds, but sometimes its 40.
Based on the log wording I dont understand what exactly is going on in
between. Maybe someone with some on-hands experience with RocksDB might
have some insights.

Thanks,
Gyula

Mime
View raw message