hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ste...@apache.org
Subject [22/31] hadoop git commit: HDFS-11433. Document missing usages of OfflineEditsViewer processors. Contributed by Yiqun Lin.
Date Mon, 27 Feb 2017 15:15:18 GMT
HDFS-11433. Document missing usages of OfflineEditsViewer processors. Contributed by Yiqun
Lin.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/4a58870a
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/4a58870a
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/4a58870a

Branch: refs/heads/HADOOP-13345
Commit: 4a58870a0427ecdf4c5c895088b96a0b4d1c0c19
Parents: e24ed47
Author: Yiqun Lin <yqlin@apache.org>
Authored: Sat Feb 25 10:00:05 2017 +0800
Committer: Yiqun Lin <yqlin@apache.org>
Committed: Sat Feb 25 10:00:05 2017 +0800

----------------------------------------------------------------------
 .../src/site/markdown/HdfsEditsViewer.md        | 73 +++++++++++++++++++-
 1 file changed, 72 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/4a58870a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md
index 7a46eb0..5e069bb 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md
@@ -40,9 +40,80 @@ The Offline Edits Viewer provides several output processors (unless stated
other
 Usage
 -----
 
+### XML Processor
+
+XML processor can create an XML file that contains the edits log information. Users can specify
input and output file via -i and -o command-line.
+
+       bash$ bin/hdfs oev -p xml -i edits -o edits.xml
+
+XML processor is the default processor in Offline Edits Viewer, users can also use the following
command:
+
        bash$ bin/hdfs oev -i edits -o edits.xml
 
-|                                       Flag | Description |
+This would result in the following output:
+
+       <?xml version="1.0" encoding="UTF-8"?>
+       <EDITS>
+         <EDITS_VERSION>-64</EDITS_VERSION>
+         <RECORD>
+           <OPCODE>OP_START_LOG_SEGMENT</OPCODE>
+           <DATA>
+             <TXID>1</TXID>
+           </DATA>
+         </RECORD>
+         <RECORD>
+           <OPCODE>OP_UPDATE_MASTER_KEY</OPCODE>
+           <DATA>
+             <TXID>2</TXID>
+             <DELEGATION_KEY>
+               <KEY_ID>1</KEY_ID>
+               <EXPIRY_DATE>1487921580728</EXPIRY_DATE>
+               <KEY>2e127ca41c7de215</KEY>
+             </DELEGATION_KEY>
+           </DATA>
+         </RECORD>
+         <RECORD>
+       ...remaining output omitted...
+
+### Binary Processor
+
+Binary processor is the opposite of the XML processor. Users can specify input XML file and
output file via -i and -o command-line.
+
+       bash$ bin/hdfs oev -p binary -i edits.xml -o edits
+
+This will reconstruct an edits log file from an XML file.
+
+### Stats Processor
+
+Stats processor is used to aggregate counts of op codes contained in the edits log file.
Users can specify this processor by -p option.
+
+       bash$ bin/hdfs oev -p stats -i edits -o edits.stats
+
+The output result of this processor should be like the following output:
+
+       VERSION                             : -64
+       OP_ADD                         (  0): 8
+       OP_RENAME_OLD                  (  1): 1
+       OP_DELETE                      (  2): 1
+       OP_MKDIR                       (  3): 1
+       OP_SET_REPLICATION             (  4): 1
+       OP_DATANODE_ADD                (  5): 0
+       OP_DATANODE_REMOVE             (  6): 0
+       OP_SET_PERMISSIONS             (  7): 1
+       OP_SET_OWNER                   (  8): 1
+       OP_CLOSE                       (  9): 9
+       OP_SET_GENSTAMP_V1             ( 10): 0
+       ...some output omitted...
+       OP_APPEND                      ( 47): 1
+       OP_SET_QUOTA_BY_STORAGETYPE    ( 48): 1
+       OP_INVALID                     ( -1): 0
+
+The output is formatted as a colon separated two column table: OpCode and OpCodeCount. Each
OpCode corresponding to the specific operation(s) in NameNode.
+
+Options
+-------
+
+| Flag | Description |
 |:---- |:---- |
 | [`-i` ; `--inputFile`] *input file* | Specify the input edits log file to process. Xml
(case insensitive) extension means XML format otherwise binary format is assumed. Required.
|
 | [`-o` ; `--outputFile`] *output file* | Specify the output filename, if the specified output
processor generates one. If the specified file already exists, it is silently overwritten.
Required. |


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message