hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1717) Enable offline deletion of entries in leveldb timeline store
Date Fri, 14 Mar 2014 18:28:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935409#comment-13935409

Billie Rinaldi commented on YARN-1717:

In testing of writing and age off at the same time, the deletion thread did not seem to adversely
affect the write rate.  With a single writer, I saw about 450 single-entity puts per second,
which is comparable to what I had observed previously.  I configured the deletion thread to
age data off after 90 seconds, and also set the deletion cycle interval to 90 seconds.  It
was able to age off data around 4500 entities per second.  With these settings, it typically
aged off on the order of 36,000 entities per cycle in less than 8 seconds.

> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>                 Key: YARN-1717
>                 URL: https://issues.apache.org/jira/browse/YARN-1717
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>         Attachments: YARN-1717.1.patch, YARN-1717.10.patch, YARN-1717.11.patch, YARN-1717.2.patch,
YARN-1717.3.patch, YARN-1717.4.patch, YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch,
YARN-1717.7.patch, YARN-1717.8.patch, YARN-1717.9.patch
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities

This message was sent by Atlassian JIRA

View raw message