hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Eagles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3978) Configurably turn off the saving of container info in Generic AHS
Date Fri, 31 Jul 2015 16:01:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649403#comment-14649403

Jonathan Eagles commented on YARN-3978:

[~eepayne], this patch looks mainly good. This feature will be good going forward for those
who want a scalable generic history solution until timeline server v2 is available. Couple
of minor things.

* It is not evident to me the changes made to TestClientRMService, TestChildQueueOrder, TestLeafQueue,
TestReservations, and TestFifoScheduler. Could you help to explain those changes or remove
them if they are extra?
* Please update the comment
+    // Store system metrics for all containers only when storeContainerMetaInfo
+    // is true.
To indicate that AM metrics publishing are delayed until later in this scenario.

* Is there a better configuration name that could be used? *save-container-meta-info* doesn't
convey that AM container info is still published if this flag is disabled.

> Configurably turn off the saving of container info in Generic AHS
> -----------------------------------------------------------------
>                 Key: YARN-3978
>                 URL: https://issues.apache.org/jira/browse/YARN-3978
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: timelineserver, yarn
>    Affects Versions: 2.8.0, 2.7.1
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>         Attachments: YARN-3978.001.patch, YARN-3978.002.patch, YARN-3978.003.patch
> Depending on how each application's metadata is stored, one week's worth of data stored
in the Generic Application History Server's database can grow to be almost a terabyte of local
disk space. In order to alleviate this, I suggest that there is a need for a configuration
option to turn off saving of non-AM container metadata in the GAHS data store.

This message was sent by Atlassian JIRA

View raw message