hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4376) Memory Timeline Store return incorrect results on fromId paging
Date Tue, 26 Jan 2016 22:27:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118184#comment-15118184
] 

Jason Lowe commented on YARN-4376:
----------------------------------

Patch looks ok -- do you have any performance numbers?  Wondering how expensive it is to maintain
the treeset.  Also this will need to be reconciled with the proposed changes in YARN-4219.
 I believe that proposed change also fixes the issue, although it's creating the treeset on
demand which could be slow for answering getEntities queries on a large dataset.

I think it's straightforward to reconcile, just need explicit valueSetIterator overrides in
the memory timeline store map adapters.

> Memory Timeline Store return incorrect results on fromId paging
> ---------------------------------------------------------------
>
>                 Key: YARN-4376
>                 URL: https://issues.apache.org/jira/browse/YARN-4376
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>         Attachments: YARN-4376.2.patch
>
>
> As pointed out correctly by [~jlowe]. 
> https://issues.apache.org/jira/browse/TEZ-2628?focusedCommentId=14715831&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14715831
> The MemoryTimelineStore cannot page correctly when using fromId. This is due switching
between data structures that apparently have different natural sorting. In addition, the approach
of creating a new data structure every time from scratch is costly. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message