hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1717) Enable offline deletion of entries in leveldb timeline store
Date Thu, 13 Mar 2014 21:23:50 GMT

     [ https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Billie Rinaldi updated YARN-1717:

    Attachment: YARN-1717.10.patch

[~zjshen], thanks for the review.  I have implemented your suggestions in the attached patch,
with the following notes.

bq. 2. Should these aging mechanism related configs have a leveldb section in the config name?
Because they're only related to the leveldb impl.
I moved ttl-interval-ms to the leveldb section, but kept ttl-ms and ttl-enable in the timeline
store section since I think those could be useful for all stores.

bq. 5. It seems not necessary to refactor "getEntity" into two methods, doesn't it?
Thanks for pointing this out.  I was able to remove a number of changes that were only needed
for the old deletion strategy.

bq. 7. In discardOldEntities, if one IOException happens, is it good to move on with the following
discarding operations?
I added a catch for the exception, logged an error, and continued deletions for the next entity

> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>                 Key: YARN-1717
>                 URL: https://issues.apache.org/jira/browse/YARN-1717
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>         Attachments: YARN-1717.1.patch, YARN-1717.10.patch, YARN-1717.2.patch, YARN-1717.3.patch,
YARN-1717.4.patch, YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch, YARN-1717.7.patch,
YARN-1717.8.patch, YARN-1717.9.patch
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities

This message was sent by Atlassian JIRA

View raw message