asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Murtadha Hubail (Code Review)" <do-not-re...@asterix-gerrit.ics.uci.edu>
Subject Change in asterixdb[master]: Allow lazy loading for persistent local resources
Date Sat, 29 Aug 2015 19:36:43 GMT
Murtadha Hubail has posted comments on this change.

Change subject: Allow lazy loading for persistent local resources
......................................................................


Patch Set 5:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/344/5/asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
File asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/PersistentLocalResourceRepository.java:

Line 205:     public HashMap<Long, LocalResource> loadAndGetAllResources() throws HyracksDataException
{
> In my opinion, a disk-access-transparent ResourceCache would serve the same
During recovery, the only piece of information that you have from the log is the resource
id. You wouldn't know where to look to begin with. Also, while the system is recovering, all
the memory is dedicated for that. There are no concurrent operations. So the memory issue
isn't likely to happen.


Line 205:     public HashMap<Long, LocalResource> loadAndGetAllResources() throws HyracksDataException
{
> Is it possible to have yet-another resourcesCache for that maps an ID to th
Yes, we would have to persist a new file at the root of every IO Device and load these files
gradually during recovery until the required resource is found. Another option is to store
a file per resource in a common location and make each file name to be the resource id. Then
from that common location, you would get where the actual resource is.

As I mentioned, I don't think this is going to be an issue during recovery, and when it does,
we could make loading the resources to the cache gradual which will make the recovery slower,
but limit the amount of memory used.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/344
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I48b9260a3280750145f6ddb3783673a299055910
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hubailmor@gmail.com>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hubailmor@gmail.com>
Gerrit-Reviewer: Yingyi Bu <buyingyi@gmail.com>
Gerrit-Reviewer: Yingyi Bu <yingyib@google.com>
Gerrit-Reviewer: Young-Seok Kim <kisskys@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-HasComments: Yes

Mime
View raw message