asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ildar Absalyamov (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: [NO ISSUE][STO][IDX] LSM storage cleanup
Date Tue, 19 Sep 2017 23:14:46 GMT
Ildar Absalyamov has posted comments on this change.

Change subject: [NO ISSUE][STO][IDX] LSM storage cleanup
......................................................................


Patch Set 16:

(4 comments)

https://asterix-gerrit.ics.uci.edu/#/c/2014/16/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMWithBuddyDiskComponent.java
File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMWithBuddyDiskComponent.java:

PS16, Line 81: createIndexWithBuddyBulkLoader
> What is the purpose of this method since we already have createComponentBul
You're right it's just an axillary method.
However, I overlooked that the same effect could be achieved just by overriding createLSMIndexBulkLoader
method form ILSMDiskComponent.
Check the new patch set.


https://asterix-gerrit.ics.uci.edu/#/c/2014/16/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/IChainedComponentBulkLoader.java
File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/IChainedComponentBulkLoader.java:

PS16, Line 26: IChainedComponentBulkLoader
> Do we need such a bulk loader with special name "chained" here? Based on it
I believe we do, this interface provides a different abstraction: add & delete methods
_returrn_ a tuple in addition to just receiving it. The returned value could be a reference
to a new tuple which is passed further down the chain. This is done to reflect a situation
when a tuple is modified (valid case for LSM filters & future Wail's schema inference


https://asterix-gerrit.ics.uci.edu/#/c/2014/16/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/api/IInPlaceInvertedIndex.java
File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/api/IInPlaceInvertedIndex.java:

PS16, Line 52: 
> It seems this method is lifted to IIndex. Any motivation to do so?
Ian had the similar question. Explained the motivation under his comment.


https://asterix-gerrit.ics.uci.edu/#/c/2014/16/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/IIndex.java
File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/IIndex.java:

PS16, Line 89: Purge the index files out of the buffer cache.
             :      * Can only be called if the caller is absolutely sure the files don't
contain dirty pages
             :      *
             :      * @throws HyracksDataException
             :      *             if the index is active
             :      */
             :     void purge() throws HyracksDataException;
             : 
             :     /**
> I'm not really sure this should be in IIndex. Only an LSM index would ever 
On contrary, this is rather a property of data structure that manages its persistent state
through buffer cache: for example, bloomFilter has a similar method.
Right now the only implementations that actually use this method are BTree(non-lsm), RTree(non-lsm)
& OnDiskInvertedInxex. Moved this method here because it's the only common interface between
those 3.
I guess the alternative is to introduce another interface.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2014
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6739d751b990e7a28e03e32a5de6e2b670d37a1e
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ildar Absalyamov <ildar.absalyamov@gmail.com>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Ildar Absalyamov <ildar.absalyamov@gmail.com>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Luo Chen <cluo8@uci.edu>
Gerrit-Reviewer: Wail Alkowaileet <wael.y.k@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-HasComments: Yes

Mime
View raw message