Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 13 Mar 2014 21:23:50 +0000 (UTC)
From: "Billie Rinaldi (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12694621.1392161985733.63506.1394745830829@arcas>
In-Reply-To: <JIRA.12694621.1392161985733@arcas>
References: <JIRA.12694621.1392161985733@arcas>
Subject: [jira] [Updated] (YARN-1717) Enable offline deletion of entries in
 leveldb timeline store
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Billie Rinaldi updated YARN-1717:
---------------------------------

    Attachment: YARN-1717.10.patch

[~zjshen], thanks for the review.  I have implemented your suggestions in the attached patch, with the following notes.

bq. 2. Should these aging mechanism related configs have a leveldb section in the config name? Because they're only related to the leveldb impl.
I moved ttl-interval-ms to the leveldb section, but kept ttl-ms and ttl-enable in the timeline store section since I think those could be useful for all stores.

bq. 5. It seems not necessary to refactor "getEntity" into two methods, doesn't it?
Thanks for pointing this out.  I was able to remove a number of changes that were only needed for the old deletion strategy.

bq. 7. In discardOldEntities, if one IOException happens, is it good to move on with the following discarding operations?
I added a catch for the exception, logged an error, and continued deletions for the next entity type.

> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>
>                 Key: YARN-1717
>                 URL: https://issues.apache.org/jira/browse/YARN-1717
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>         Attachments: YARN-1717.1.patch, YARN-1717.10.patch, YARN-1717.2.patch, YARN-1717.3.patch, YARN-1717.4.patch, YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch, YARN-1717.7.patch, YARN-1717.8.patch, YARN-1717.9.patch
>
>
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities


--
This message was sent by Atlassian JIRA
(v6.2#6252)