hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bianqi (Jira)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-15364) Sort the output according to the number of occurrences of the opcode for StatisticsEditsVisitor
Date Tue, 19 May 2020 16:45:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-15364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

bianqi updated HDFS-15364:
--------------------------
    Status: Open  (was: Patch Available)

> Sort the output according to the number of occurrences of the opcode for StatisticsEditsVisitor
> -----------------------------------------------------------------------------------------------
>
>                 Key: HDFS-15364
>                 URL: https://issues.apache.org/jira/browse/HDFS-15364
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: tools
>    Affects Versions: 3.2.1
>            Reporter: bianqi
>            Assignee: bianqi
>            Priority: Minor
>         Attachments: HDFS-15364.001.patch
>
>
>       At present, when we execute `hdfs oev -p stats -i edits -o edits.stats`, the
output format is as follows, and all the opcodes will be output once.
> {quote}VERSION : -65
>  OP_ADD ( 0): 2
>  OP_RENAME_OLD ( 1): 2
>  OP_DELETE ( 2): 0
>  OP_MKDIR ( 3): 5
>  OP_SET_REPLICATION ( 4): 0
>  OP_DATANODE_ADD ( 5): 0
>  OP_DATANODE_REMOVE ( 6): 0
>  OP_SET_PERMISSIONS ( 7): 4
>  OP_SET_OWNER ( 8): 1
>  OP_CLOSE ( 9): 2
>  OP_SET_GENSTAMP_V1 ( 10): 0
>  OP_SET_NS_QUOTA ( 11): 0
>  OP_CLEAR_NS_QUOTA ( 12): 0
>  OP_TIMES ( 13): 0
>  OP_SET_QUOTA ( 14): 0
>  OP_RENAME ( 15): 0
>  OP_CONCAT_DELETE ( 16): 0
>  OP_SYMLINK ( 17): 0
>  OP_GET_DELEGATION_TOKEN ( 18): 0
>  OP_RENEW_DELEGATION_TOKEN ( 19): 0
>  OP_CANCEL_DELEGATION_TOKEN ( 20): 0
>  OP_UPDATE_MASTER_KEY ( 21): 0
>  OP_REASSIGN_LEASE ( 22): 0
>  OP_END_LOG_SEGMENT ( 23): 1
>  OP_START_LOG_SEGMENT ( 24): 1
>  OP_UPDATE_BLOCKS ( 25): 0
>  OP_CREATE_SNAPSHOT ( 26): 0
>  OP_DELETE_SNAPSHOT ( 27): 0
>  OP_RENAME_SNAPSHOT ( 28): 0
>  OP_ALLOW_SNAPSHOT ( 29): 0
>  OP_DISALLOW_SNAPSHOT ( 30): 0
>  OP_SET_GENSTAMP_V2 ( 31): 2
>  OP_ALLOCATE_BLOCK_ID ( 32): 2
>  OP_ADD_BLOCK ( 33): 2
>  OP_ADD_CACHE_DIRECTIVE ( 34): 0
>  OP_REMOVE_CACHE_DIRECTIVE ( 35): 0
>  OP_ADD_CACHE_POOL ( 36): 0
>  OP_MODIFY_CACHE_POOL ( 37): 0
>  OP_REMOVE_CACHE_POOL ( 38): 0
>  OP_MODIFY_CACHE_DIRECTIVE ( 39): 0
>  OP_SET_ACL ( 40): 0
>  OP_ROLLING_UPGRADE_START ( 41): 0
>  OP_ROLLING_UPGRADE_FINALIZE ( 42): 0
>  OP_SET_XATTR ( 43): 0
>  OP_REMOVE_XATTR ( 44): 0
>  OP_SET_STORAGE_POLICY ( 45): 0
>  OP_TRUNCATE ( 46): 0
>  OP_APPEND ( 47): 0
>  OP_SET_QUOTA_BY_STORAGETYPE ( 48): 0
>  OP_ADD_ERASURE_CODING_POLICY ( 49): 0
>  OP_ENABLE_ERASURE_CODING_POLIC ( 50): 0
>  OP_DISABLE_ERASURE_CODING_POLI ( 51): 0
>  OP_REMOVE_ERASURE_CODING_POLIC ( 52): 0
>  OP_INVALID ( -1): 0
> {quote}
>  But in general, the edits file we parse does not involve all the operation codes. If
all the operation codes are output, it is unfriendly for the cluster administrator to view
the output.
>     we usually only care about what opcodes appear in the edits file.We can output
the opcodes that appeared in the edits file and sort them.
> For example, we can execute the following command:
> {quote} hdfs oev -p stats -i edits_0000000000000001321-0000000000000001344 -sort -o
edits.stats -v
> {quote}
> The output format is as follows:
> {quote}VERSION : -65
>  OP_MKDIR ( 3): 5
>  OP_SET_PERMISSIONS ( 7): 4
>  OP_ADD ( 0): 2
>  OP_RENAME_OLD ( 1): 2
>  OP_CLOSE ( 9): 2
>  OP_SET_GENSTAMP_V2 ( 31): 2
>  OP_ALLOCATE_BLOCK_ID ( 32): 2
>  OP_ADD_BLOCK ( 33): 2
>  OP_SET_OWNER ( 8): 1
>  OP_END_LOG_SEGMENT ( 23): 1
>  OP_START_LOG_SEGMENT ( 24): 1
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message