hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log
Date Thu, 18 Oct 2018 18:35:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655716#comment-16655716

Jonathan Hung commented on YARN-7225:

Hi [~eepayne] thanks for updating the patch.

Sorry I missed this in the previous review. But I think the logged partition should be changed
from logging the partition of the AM container request to the partition the container ran

for the logSuccess in FiCaSchedulerApp#containerCompleted:
// In order to save space in the audit log, only include the partition
// if it is not the default partition.
String containerPartition = null;
if (appAMNodePartitionName != null &&
      !appAMNodePartitionName.isEmpty()) {
  containerPartition = appAMNodePartitionName;
Resource containerResource = rmContainer.getContainer().getResource();
RMAuditLogger.logSuccess(getUser(), AuditConstants.RELEASE_CONTAINER,
    "SchedulerApp", getApplicationId(), containerId, containerResource,
    getQueueName(), containerPartition);
We should just be able to log "partition" instead of containerPartition since the partition
passed to containerCompleted is the node's partition

For the logSuccess inĀ  FiCaScheduler#apply:
String partition = null;
if (appAMNodePartitionName != null &&
      !appAMNodePartitionName.isEmpty()) {
  partition = appAMNodePartitionName;
RMAuditLogger.logSuccess(getUser(), AuditConstants.ALLOC_CONTAINER,
    "SchedulerApp", getApplicationId(), containerId,
    allocation.getAllocatedOrReservedResource(), getQueueName(),
We should be able to log schedulerContainer.getSchedulerNode().getPartition.

> Add queue and partition info to RM audit log
> --------------------------------------------
>                 Key: YARN-7225
>                 URL: https://issues.apache.org/jira/browse/YARN-7225
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.9.1, 2.8.4, 3.0.2, 3.1.1
>            Reporter: Jonathan Hung
>            Assignee: Eric Payne
>            Priority: Major
>         Attachments: YARN-7225.001.patch, YARN-7225.002.patch, YARN-7225.003.patch, YARN-7225.004.patch,
> Right now RM audit log has fields such as user, ip, resource, etc. Having queue and partition
 is useful for resource tracking.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message