hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4851) Metric improvements for ATS v1.5 storage
Date Thu, 07 Apr 2016 22:17:25 GMT

     [ https://issues.apache.org/jira/browse/YARN-4851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Li Lu updated YARN-4851:
------------------------
    Attachment: YARN-4851-trunk.001.patch

First draft of the patch. In this patch I've added some metrics for the ATS v1.5 storage.
Specifically:

For overall system usage:
- Number of read requests to summary storage
- Number of read requests to detail storage
- Number of entities scanned by EntityGroupFS storage into the summary storage. 
- Accumulated time spent on scanning new apps in the active directory
- Accumulated time spent on reading summary data into the summary storage

Caching performance:
- Number of cache storage refreshes (cache reloads). This can be compared to the number of
read requests to detail storage to understand how useful a caching layer is for specific cluster
workload. 
- Accumulated time spent on refreshing cache storages. 

Log cleaner/purging:
- Number of dirs purged by the storage
- Accumulated time for log cleaning. 

> Metric improvements for ATS v1.5 storage
> ----------------------------------------
>
>                 Key: YARN-4851
>                 URL: https://issues.apache.org/jira/browse/YARN-4851
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-4851-trunk.001.patch
>
>
> We can add more metrics to the ATS v1.5 storage systems, including purging, cache hit/misses,
read latency, etc. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message