hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-925) HistoryStorage Reader Interface for Application History Server
Date Mon, 21 Oct 2013 18:29:47 GMT

    [ https://issues.apache.org/jira/browse/YARN-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800928#comment-13800928

Zhijie Shen commented on YARN-925:

bq. if you look at the filters more closely you will see we have offset and limit in the filters
which will actually gives you iterative functionality , however client has to call them with
the right offset.

I thought you agree that we should be able to return all applications, right? Let's assume
we have 1 M applications in store, do we really want to create a map that contains 1 M applications
and return? I'm afraid it will explode the memory. That's why I think we should return Iterator
instead, just like what JDBC does (e.g. ResultSet). It's not about the filter, I mean anyway
we may be in the situation of returning a huge number of applications.

> HistoryStorage Reader Interface for Application History Server
> --------------------------------------------------------------
>                 Key: YARN-925
>                 URL: https://issues.apache.org/jira/browse/YARN-925
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Mayank Bansal
>            Assignee: Mayank Bansal
>             Fix For: YARN-321
>         Attachments: YARN-925-1.patch, YARN-925-2.patch, YARN-925-3.patch, YARN-925-4.patch,
YARN-925-5.patch, YARN-925-6.patch

This message was sent by Atlassian JIRA

View raw message