asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luo Chen (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Avoid always merging old components in prefix policy
Date Tue, 13 Jun 2017 23:35:23 GMT
Luo Chen has posted comments on this change.

Change subject: Avoid always merging old components in prefix policy
......................................................................


Patch Set 7:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1818/6/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java
File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java:

PS6, Line 273:          }
> What I proposed is almost the same as what you're doing.  Since the current
I see. But I suspect that this new change wouldn't bring too much improvement to the current
patch, but instead making it more complicated (right now the logic is already a bit complicated...)

Still, I think this idea is a specialization of the level-based policy, where all flushed
disk components have level 0, but merged disk components have level 1 (here we don't differentiate
between multiple rounds of merges).

Moreover, if consider the layout of the disk components resulting from this policy, then the
actual checking wound't takes too much time. Suppose MaxMergableSize = 100M, MaxToleranceCount
= 5, initial flushed disk component has size 1M, then the "worst" layout of 100 disk components
would be like:

>100M, >100M , ..., >100M, 61M, 25M, 11M, 5M, 1M, 1M, 1M, 1M

Here the prefix policy wound't schedule any merges. But when a new disk component is added,
the policy would first skip all too large components, and only consider the newest (smallest)
components. The size the mergeable components considered by the policy is always small, and
it actually has some upper bound.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1818
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen <cluo8@uci.edu>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Jianfeng Jia <jianfeng.jia@gmail.com>
Gerrit-Reviewer: Luo Chen <cluo8@uci.edu>
Gerrit-Reviewer: Yingyi Bu <buyingyi@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-HasComments: Yes

Mime
View raw message