hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log
Date Thu, 18 Oct 2018 18:35:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655716#comment-16655716
] 

Jonathan Hung commented on YARN-7225:
-------------------------------------

Hi [~eepayne] thanks for updating the patch.

Sorry I missed this in the previous review. But I think the logged partition should be changed
from logging the partition of the AM container request to the partition the container ran
on:

for the logSuccess in FiCaSchedulerApp#containerCompleted:
{noformat}
// In order to save space in the audit log, only include the partition
// if it is not the default partition.
String containerPartition = null;
if (appAMNodePartitionName != null &&
      !appAMNodePartitionName.isEmpty()) {
  containerPartition = appAMNodePartitionName;
}
Resource containerResource = rmContainer.getContainer().getResource();
RMAuditLogger.logSuccess(getUser(), AuditConstants.RELEASE_CONTAINER,
    "SchedulerApp", getApplicationId(), containerId, containerResource,
    getQueueName(), containerPartition);
{noformat}
We should just be able to log "partition" instead of containerPartition since the partition
passed to containerCompleted is the node's partition

For the logSuccess inĀ  FiCaScheduler#apply:
{noformat}
String partition = null;
if (appAMNodePartitionName != null &&
      !appAMNodePartitionName.isEmpty()) {
  partition = appAMNodePartitionName;
}
RMAuditLogger.logSuccess(getUser(), AuditConstants.ALLOC_CONTAINER,
    "SchedulerApp", getApplicationId(), containerId,
    allocation.getAllocatedOrReservedResource(), getQueueName(),
    partition);{noformat}
We should be able to log schedulerContainer.getSchedulerNode().getPartition.

> Add queue and partition info to RM audit log
> --------------------------------------------
>
>                 Key: YARN-7225
>                 URL: https://issues.apache.org/jira/browse/YARN-7225
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.9.1, 2.8.4, 3.0.2, 3.1.1
>            Reporter: Jonathan Hung
>            Assignee: Eric Payne
>            Priority: Major
>         Attachments: YARN-7225.001.patch, YARN-7225.002.patch, YARN-7225.003.patch, YARN-7225.004.patch,
YARN-7225.branch-2.8.001.patch
>
>
> Right now RM audit log has fields such as user, ip, resource, etc. Having queue and partition
 is useful for resource tracking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message