asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luo Chen (Code Review)" <>
Subject Change in asterixdb[master]: Avoid always merging old components in prefix policy
Date Fri, 09 Jun 2017 03:34:16 GMT
Luo Chen has uploaded a new patch set (#2).

Change subject: Avoid always merging old components in prefix policy

Avoid always merging old components in prefix policy

Current, the prefix policy always looks at the components from
oldest to newest to schedule merge. One negative consequence is that
the oldest (largest) component gets merged over and over again
until it reaches the size limit. This is undesirable since it takes
O(n^2) disk IOs (n is the number of flushed components) to produce a
final component.

This patch is a temporary fix of this behavior, taken from the idea of
HBase compaction policy (
-flushes-and-compactions/). The basic idea is that it introduces
some size factor (for now it's 1.2) to control the merge behavior.

When the prefix policy finds a sequence of components to merge,
we also check the oldest (largest) component in the sequence should
be smaller than 1.2*the total size of all younger components.
By doing so, we can avoid merging oldest components over and over again,
making the disk IOs O(nlog n).

Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/pom.xml
A hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common/test/
3 files changed, 247 insertions(+), 37 deletions(-)

  git pull ssh:// refs/changes/18/1818/2
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen <>
Gerrit-Reviewer: Jenkins <>

View raw message