hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1717) Enable offline deletion of entries in leveldb timeline store
Date Fri, 07 Mar 2014 17:21:44 GMT

    [ https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924072#comment-13924072

Billie Rinaldi commented on YARN-1717:

bq. 1. How can users invoke the deletion operation?

A command like the following should remove all entries before and including the given timestamp.
 (The timeline service can't be running at the same time.)
bin/yarn org.apache.hadoop.yarn.server.applicationhistoryservice.timeline.LeveldbTimelineStore\$DeleteOldEntries

bq. Is it better to automatically discard the entities whose timestamp is older than the configured
timestamp? At lease, it could be an option other than manually deleting the entities. Thoughts?

Certainly online deletion would be better than offline, though that is beyond the stated scope
of this ticket.  In my opinion the best first step in that direction would be to add a deleteEntity
request to the API, but the actual deletion would still be manual.  To automatically discard
old entities, we'd have to do something like add a thread that runs the deleteOldEntries method
periodically, and we'd have to make sure it wouldn't affect performance too much.

bq. I'm a bit concerned of returning EntityWithReverseRelatedEntities from the leveldb timeline
store, because it will potentially change the output schema.

I don't understand how it would change the output schema (since it extends TimelineEntity
and doesn't add any XmlElements).  Can you explain?

> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>                 Key: YARN-1717
>                 URL: https://issues.apache.org/jira/browse/YARN-1717
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>         Attachments: YARN-1717.1.patch, YARN-1717.2.patch, YARN-1717.3.patch, YARN-1717.4.patch,
YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch, YARN-1717.7.patch, YARN-1717.8.patch
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities

This message was sent by Atlassian JIRA

View raw message