asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luo Chen (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Implemented Disk Components Alignment Based on IDs
Date Wed, 24 May 2017 04:35:47 GMT
Luo Chen has posted comments on this change.

Change subject: Implemented Disk Components Alignment Based on IDs
......................................................................


Patch Set 6:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1761/6/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicy.java
File asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/CorrelatedPrefixMergePolicy.java:

PS6, Line 103: isMergeLagging(
> I am strongly opposed to delay of flushes for the sake of merges. Not sure 
Both solutions (with or without flow control) are solution OK to me. But without flow control,
one consequence is that there could be more and more lagged disk components accumulated on
the disk along with the data ingestion. But eventually when the ingestion stops, the disk
components would be merged as expected. Any suggestions or comments from other reviewers?

However, for the correlated merge policy, the merge operation shouldn't be triggered by any
secondary index. The reason is that the secondary index disk component is usually much smaller
than the primary index disk component. If the merge request is sent by the secondary index
(for all indexes), then we would get much larger disk components for the primary, which breaks
the criteria for the merge policy. For example, if we set the MaxMergableComponentSize=128M
(this tells the merge policy that if any disk component is larger than this size, then never
merge it again), when the primary index disk component reaches this limit, the corresponding
secondary index disk component is only about 4-5M.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1761
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I768ee9ac0a8d3c99c631086093a6b778b2e7588e
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen <cluo8@uci.edu>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Jianfeng Jia <jianfeng.jia@gmail.com>
Gerrit-Reviewer: Luo Chen <cluo8@uci.edu>
Gerrit-Reviewer: Till Westmann <tillw@apache.org>
Gerrit-Reviewer: Yingyi Bu <yingyib@google.com>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-HasComments: Yes

Mime
View raw message