hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1717) Enable offline deletion of entries in leveldb timeline store
Date Fri, 14 Mar 2014 15:14:51 GMT

    [ https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935128#comment-13935128

Billie Rinaldi commented on YARN-1717:

bq. We do deletion according to entity's TS and at the entity's granularity, thus, the events
that are still alive are likely to be deleted as well.

I believe this is the desired behavior.  For example, in the case where we have a job entity
that starts several shorter-lived task entities, we would not want to remove task entities
before the job entity is removed.  With the current behavior, the job entity would be removed
at the same time or earlier than the task entities.

We don't yet have a good understanding of how applications with long-lived entities would
want to use the timeline store, so it's hard to design for them.  Perhaps an option for the
future would be to have a configurable deletion strategy, if some applications have different

> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>                 Key: YARN-1717
>                 URL: https://issues.apache.org/jira/browse/YARN-1717
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>         Attachments: YARN-1717.1.patch, YARN-1717.10.patch, YARN-1717.11.patch, YARN-1717.2.patch,
YARN-1717.3.patch, YARN-1717.4.patch, YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch,
YARN-1717.7.patch, YARN-1717.8.patch, YARN-1717.9.patch
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities

This message was sent by Atlassian JIRA

View raw message