phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Soldatov (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3871) Incremental stats collection
Date Mon, 22 May 2017 23:05:04 GMT


Sergey Soldatov commented on PHOENIX-3871:

We collect statistic on all compactions that have COMPACT_DROP_DELETES. Those not only major
compactions, but also minor compactions if one of default compaction policies are used ( RatioBasedCompactionPolicy
sets the compaction as major if all storefile candidates get into the compaction). Running
statistic collection on upserts sounds like an overkill. 

> Incremental stats collection
> ----------------------------
>                 Key: PHOENIX-3871
>                 URL:
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Eli Levine
> Phoenix automatically gathers statistics at [major compaction time|].
While this is useful and accurate, it also means that statistics can become stale due to the
infrequency of major compactions (can be days between major compactions), reducing their usefulness.

> This jira asks the question: Is it possible for Phoenix to collects statistics at a more
granular level, say for every (or a sampling of) UPSERT, or minor compaction. Since statistics
are always approximations, it is OK for this incremental approach to not be 100% accurate.
> The current stats collection mechanism at major compaction time should be kept to accurately
"fix up" stats at major compaction time.
> [~jamestaylor], FYI. We talked about this in person a few weeks ago. Creating this Jira
for posterity. Please add anything that I missed. Thanks!

This message was sent by Atlassian JIRA

View raw message