hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1455) Record DFS client/cli id with username/kerbros session token in audit log or hdfs client trace log
Date Wed, 13 Oct 2010 21:32:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920774#action_12920774
] 

Eric Yang commented on HDFS-1455:
---------------------------------

Correct, this is for reporting purpose.  It is possible to stream hdfs client trace through
syslog protocol to chukwa to get near real-time analysis.

> Record DFS client/cli id with username/kerbros session token in audit log or hdfs client
trace log
> --------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1455
>                 URL: https://issues.apache.org/jira/browse/HDFS-1455
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Eric Yang
>
> HDFS usage calculation is commonly calculated by running dfs -dus and group directory
usage by user at fix interval.  This approach does not show accurate HDFS usage if a lot of
read/write activity of equivalent amount of data happen at fix interval.  In order to identify
usage of such pattern, the usage calculation could be measured by the bytes read and bytes
written in the hdfs client trace log.  There is currently no association of DFSClient ID or
CLI ID to the user or session token emitted by Hadoop hdfs client trace log files.  This JIRA
is to record DFS Client ID/CLI ID with user name/session token in appropriate place for more
precious measuring of HDFS usage.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message