accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ctubb...@apache.org
Subject [1/5] ACCUMULO-1490 Move html and config docs to monitor
Date Tue, 24 Sep 2013 23:00:17 GMT
Updated Branches:
  refs/heads/master 583453721 -> acba59b16


http://git-wip-us.apache.org/repos/asf/accumulo/blob/acba59b1/server/src/main/resources/docs/metrics.html
----------------------------------------------------------------------
diff --git a/server/src/main/resources/docs/metrics.html b/server/src/main/resources/docs/metrics.html
new file mode 100644
index 0000000..00f0a5b
--- /dev/null
+++ b/server/src/main/resources/docs/metrics.html
@@ -0,0 +1,182 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+<html>
+<head>
+<title>Accumulo Metrics</title>
+<link rel='stylesheet' type='text/css' href='documentation.css' media='screen'/>
+</head>
+<body>
+
+<h1>Apache Accumulo Documentation : Metrics</h1>
+
+As of version 1.2, metrics for the Master, Tablet Servers, and Loggers are available. A new
configuration file, accumulo-metrics.xml, is located in the conf directory and can
+be modified to turn metrics collection on or off, and to enable file logging if desired.
This file can be modified at runtime and the changes will be seen after a few seconds.
+Except where specified all time values are in milliseconds.
+<h1>Master Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=MasterMetricsMBean,name=
&lt;current thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public long getPingCount();</td><td>Number
of pings to tablet servers</td></tr>
+		<tr><td>public long getPingAvgTime();</td><td>Average time for
each ping</td></tr>
+		<tr class="highlight"><td>public long getPingMinTime();</td><td>Minimum
time for each ping</td></tr>
+		<tr><td>public long getPingMaxTime();</td><td>Maximum time for
each ping</td></tr>
+		<tr class="highlight"><td>public String getTServerWithHighestPingTime();</td><td>tablet
server with highest ping</td></tr>
+		<tr><td>public void reset();</td><td>Resets all counters to zero</td></tr>
+	</tbody>
+</table>
+<h1>Logging Server Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=LogWriterMBean,name= &lt;current
thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public long getCloseCount();</td><td>Number
of closed log files</td></tr>
+		<tr><td>public long getCloseAvgTime();</td><td>Average time to
close a log file</td></tr>
+		<tr class="highlight"><td>public long getCloseMinTime();</td><td>Minimum
time to close a log file</td></tr>
+		<tr><td>public long getCloseMaxTime();</td><td>Maximum time to
close a log file</td></tr>
+		<tr class="highlight"><td>public long getCopyCount();</td><td>Number
of log files copied</td></tr>
+		<tr><td>public long getCopyAvgTime();</td><td>Average time to copy
a log file</td></tr>
+		<tr class="highlight"><td>public long getCopyMinTime();</td><td>Minimum
time to copy a log file</td></tr>
+		<tr><td>public long getCopyMaxTime();</td><td>Maximum time to copy
a log file</td></tr>
+		<tr class="highlight"><td>public long getCreateCount();</td><td>Number
of log files created</td></tr>
+		<tr><td>public long getCreateMinTime();</td><td>Minimum time to
create a log file</td></tr>
+		<tr class="highlight"><td>public long getCreateMaxTime();</td><td>Maximum
time to create a log file</td></tr>
+		<tr><td>public long getCreateAvgTime();</td><td>Average time to
create a log file</td></tr>
+		<tr class="highlight"><td>public long getLogAppendCount();</td><td>Number
of times logs have been appended</td></tr>
+		<tr><td>public long getLogAppendMinTime();</td><td>Minimum time
to append to a log file</td></tr>
+		<tr class="highlight"><td>public long getLogAppendMaxTime();</td><td>Maximum
time to append to a log file</td></tr>
+		<tr><td>public long getLogAppendAvgTime();</td><td>Average time
to append to a log file</td></tr>
+		<tr class="highlight"><td>public long getLogFlushCount();</td><td>Number
of log file flushes</td></tr>
+		<tr><td>public long getLogFlushMinTime();</td><td>Minimum time
to flush a log file</td></tr>
+		<tr class="highlight"><td>public long getLogFlushMaxTime();</td><td>Maximum
time to flush a log file</td></tr>
+		<tr><td>public long getLogFlushAvgTime();</td><td>Average time
to flush a log file</td></tr>
+		<tr class="highlight"><td>public long getLogExceptionCount();</td><td>Number
of log exceptions</td></tr>
+		<tr><td>public void reset();</td><td>Resets all counters to zero</td></tr>
+	</tbody>
+</table>
+<h1>Tablet Server Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=TabletServerMBean,name=
&lt;current thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public int getOnlineCount();</td><td>Number
of tablets online</td></tr>
+		<tr><td>public int getOpeningCount();</td><td>Number of tablets
that are being opened</td></tr>
+		<tr class="highlight"><td>public int getUnopenedCount();</td><td>Number
or unopened tablets</td></tr>
+		<tr><td>public int getMajorCompactions();</td><td>Number of Major
Compactions currently running</td></tr>
+		<tr class="highlight"><td>public int getMajorCompactionsQueued();</td><td>Number
of Major Compactions yet to run</td></tr>
+		<tr><td>public int getMinorCompactions();</td><td>Number of Minor
Compactions currently running</td></tr>
+		<tr class="highlight"><td>public int getMinorCompactionsQueued();</td><td>Number
of Minor Compactions yet to run</td></tr>
+		<tr><td>public int getShutdownStage();</td><td>Current stage in
the shutdown process</td></tr>
+		<tr class="highlight"><td>public long getEntries();</td><td>Number
of entries in all the tablets</td></tr>
+		<tr><td>public long getEntriesInMemory();</td><td>Number of entries
in memory on all tablet servers</td></tr>
+		<tr class="highlight"><td>public long getQueries();</td><td>Number
of queries currently running on all the tablet servers</td></tr>
+		<tr><td>public long getIngest();</td><td>Number of entries currently
being ingested on all the tablet servers</td></tr>
+		<tr class="highlight"><td>public long getTotalMinorCompactions();</td><td>Number
of Minor Compactions completed</td></tr>
+		<tr><td>public double getHoldTime();</td><td>Number of seconds
that ingest is waiting for memory to be freed on tablet servers</td></tr>
+		<tr class="highlight"><td>public String getName();</td><td>Address
of the master</td></tr>
+	</tbody>
+</table>
+<h1>Tablet Server Minor Compaction Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=TabletServerMinCMetricsMBean,name=
&lt;current thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public long getMinorCompactionCount();</td><td>Number
of completed Minor Compactions on all tablet servers</td></tr>
+		<tr><td>public long getMinorCompactionAvgTime();</td><td>Average
time to complete Minor Compaction</td></tr>
+		<tr class="highlight"><td>public long getMinorCompactionMinTime();</td><td>Minimum
time to complete Minor Compaction</td></tr>
+		<tr><td>public long getMinorCompactionMaxTime();</td><td>Maximum
time to complete Minor Compaction</td></tr>
+		<tr class="highlight"><td>public long getMinorCompactionQueueCount();</td><td>Number
of Minor Compactions yet to be run</td></tr>
+		<tr><td>public long getMinorCompactionQueueAvgTime();</td><td>Average
time Minor Compaction is in the queue</td></tr>
+		<tr class="highlight"><td>public long getMinorCompactionQueueMinTime();</td><td>Minimum
time Minor Compaction is in the queue</td></tr>
+		<tr><td>public long getMinorCompactionQueueMaxTime();</td><td>Maximum
time Minor Compaction is in the queue</td></tr>
+		<tr class="highlight"><td>public void reset();</td><td>Resets all
counters to zero</td></tr>
+	</tbody>
+</table>
+<h1>Tablet Server Scan Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=TabletServerScanMetricsMBean,name=
&lt;current thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public long getScanCount();</td><td>Number
of scans completed</td></tr>
+		<tr><td>public long getScanAvgTime();</td><td>Average time for
scan operation</td></tr>
+		<tr class="highlight"><td>public long getScanMinTime();</td><td>Minimum
time for scan operation</td></tr>
+		<tr><td>public long getScanMaxTime();</td><td>Maximum time for
scan operation</td></tr>
+		<tr class="highlight"><td>public long getResultCount();</td><td>Number
of scans that returned a result</td></tr>
+		<tr><td>public long getResultAvgSize();</td><td>Average size of
scan result</td></tr>
+		<tr class="highlight"><td>public long getResultMinSize();</td><td>Minimum
size of scan result</td></tr>
+		<tr><td>public long getResultMaxSize();</td><td>Maximum size of
scan result</td></tr>
+		<tr class="highlight"><td>public void reset();</td><td>Resets all
counters to zero</td></tr>
+	</tbody>
+</table>
+<h1>Tablet Server Update Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=TabletServerUpdateMetricsMBean,name=
&lt;current thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public long getPermissionErrorCount();</td><td>Number
of permission errors</td></tr>
+		<tr><td>public long getUnknownTabletErrorCount();</td><td>Number
of unknown tablet errors</td></tr>
+		<tr class="highlight"><td>public long getMutationArrayAvgSize();</td><td>Average
size of mutation array</td></tr>
+		<tr><td>public long getMutationArrayMinSize();</td><td>Minimum
size of mutation array</td></tr>
+		<tr class="highlight"><td>public long getMutationArrayMaxSize();</td><td>Maximum
size of mutation array</td></tr>
+		<tr><td>public long getCommitPrepCount();</td><td>Number of commit
preparations</td></tr>
+		<tr class="highlight"><td>public long getCommitPrepMinTime();</td><td>Minimum
time for commit preparation</td></tr>
+		<tr><td>public long getCommitPrepMaxTime();</td><td>Maximum time
for commit preparatation</td></tr>
+		<tr class="highlight"><td>public long getCommitPrepAvgTime();</td><td>Average
time for commit preparation</td></tr>
+		<tr><td>public long getConstraintViolationCount();</td><td>Number
of constraint violations</td></tr>
+		<tr class="highlight"><td>public long getWALogWriteCount();</td><td>Number
of writes to the Write Ahead Log</td></tr>
+		<tr><td>public long getWALogWriteMinTime();</td><td>Minimum time
of a write to the Write Ahead Log</td></tr>
+		<tr class="highlight"><td>public long getWALogWriteMaxTime();</td><td>Maximum
time of a write to the Write Ahead Log</td></tr>
+		<tr><td>public long getWALogWriteAvgTime();</td><td>Average time
of a write to the Write Ahead Log</td></tr>
+		<tr class="highlight"><td>public long getCommitCount();</td><td>Number
of commits</td></tr>
+		<tr><td>public long getCommitMinTime();</td><td>Minimum time for
a commit</td></tr>
+		<tr class="highlight"><td>public long getCommitMaxTime();</td><td>Maximum
time for a commit</td></tr>
+		<tr><td>public long getCommitAvgTime();</td><td>Average time for
a commit</td></tr>
+		<tr class="highlight"><td>public void reset();</td><td>Resets all
counters to zero</td></tr>
+	</tbody>
+</table>
+<h1>Thrift Server Metrics</h1>
+<p>JMX Object Name: org.apache.accumulo.server.metrics:type=ThriftMetricsMBean,name=
&lt;thread name&gt;</p>
+<table>
+	<thead>
+		<tr><td>Method Name</td><td>Description</td></tr>
+	</thead>
+	<tbody>
+		<tr class="highlight"><td>public long getIdleCount();</td><td>Number
of times the Thrift server has been idle</td></tr>
+		<tr><td>public long getIdleMinTime();</td><td>Minimum amount of
time the Thrift server has been idle</td></tr>
+		<tr class="highlight"><td>public long getIdleMaxTime();</td><td>Maximum
amount of time the Thrift server has been idle</td></tr>
+		<tr><td>public long getIdleAvgTime();</td><td>Average time the
Thrift server has been idle</td></tr>
+		<tr class="highlight"><td>public long getExecutionCount();</td><td>Number
of calls processed by the Thrift server</td></tr>
+		<tr><td>public long getExecutionMinTime();</td><td>Minimum amount
of time executing method</td></tr>
+		<tr class="highlight"><td>public long getExecutionMaxTime();</td><td>Maximum
amount of time executing method</td></tr>
+		<tr><td>public long getExecutionAvgTime();</td><td>Average time
executing methods</td></tr>
+		<tr class="highlight"><td>public void reset();</td><td>Resets all
counters to zero</td></tr>
+	</tbody>
+</table>
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/accumulo/blob/acba59b1/server/src/main/resources/docs/timestamps.html
----------------------------------------------------------------------
diff --git a/server/src/main/resources/docs/timestamps.html b/server/src/main/resources/docs/timestamps.html
new file mode 100644
index 0000000..52290c7
--- /dev/null
+++ b/server/src/main/resources/docs/timestamps.html
@@ -0,0 +1,160 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+<html>
+<head>
+<title>Accumulo Timestamps</title>
+<link rel='stylesheet' type='text/css' href='documentation.css' media='screen'/>
+</head>
+<body>
+
+<h1>Apache Accumulo Documentation : Timestamps</h1>
+
+<p>Everything inserted into accumulo has a timestamp.  If the user does not
+set it, then the system will set the timestamp.  The timestamp is the last
+thing accumulo sorts on.  So when two keys have the same row, column family,
+column qualifier, and column visibility then the timestamp of the two keys is
+compared. 
+
+<p>Timestamps are sorted in descending order, so the most recent data comes
+first.  When a table is created in accumulo, by default it has a versioning
+iterator that only shows the most recent.  In the example below two identical
+things are inserted.  The scan after that only shows the most recent version.
+However when the versioning iterator configuration is changed, then both are
+seen.  When data is inserted with a lower timestamp than existing data, it will
+fall behind the existing data and may not be seen depending on the versioning
+settings.  This is why the insert made with a timestamp of 500 is not seen in
+the scan below.
+
+<p><pre>
+root@ac12&gt; createtable foo
+root@ac12 foo&gt; 
+root@ac12 foo&gt; 
+root@ac12 foo&gt; insert r1 cf1 cq1 value1                                   
+root@ac12 foo&gt; insert r1 cf1 cq1 value2
+root@ac12 foo&gt; scan -st
+r1 cf1:cq1 [] 1279906856203    value2
+root@ac12 foo&gt; config -t foo -f iterator                                  
+---------+---------------------------------------------+-----------------------------------------------------------------------------------------------------
+SCOPE    | NAME                                        | VALUE
+---------+---------------------------------------------+-----------------------------------------------------------------------------------------------------
+table    | table.iterator.majc.vers .................. | 20,org.apache.accumulo.core.iterators.VersioningIterator
+table    | table.iterator.majc.vers.opt.maxVersions .. | 1
+table    | table.iterator.minc.vers .................. | 20,org.apache.accumulo.core.iterators.VersioningIterator
+table    | table.iterator.minc.vers.opt.maxVersions .. | 1
+table    | table.iterator.scan.vers .................. | 20,org.apache.accumulo.core.iterators.VersioningIterator
+table    | table.iterator.scan.vers.opt.maxVersions .. | 1
+---------+---------------------------------------------+-----------------------------------------------------------------------------------------------------
+root@ac12 foo&gt; config -t foo -s table.iterator.scan.vers.opt.maxVersions=3
+root@ac12 foo&gt; config -t foo -s table.iterator.minc.vers.opt.maxVersions=3
+root@ac12 foo&gt; config -t foo -s table.iterator.majc.vers.opt.maxVersions=3
+root@ac12 foo&gt; scan -st
+r1 cf1:cq1 [] 1279906856203    value2
+r1 cf1:cq1 [] 1279906853170    value1
+root@ac12 foo&gt; insert -t 600 r1 cf1 cq1 value3
+root@ac12 foo&gt; insert -t 500 r1 cf1 cq1 value4
+root@ac12 foo&gt; scan -st
+r1 cf1:cq1 [] 1279906856203    value2
+r1 cf1:cq1 [] 1279906853170    value1
+r1 cf1:cq1 [] 600    value3
+root@ac12 foo&gt;
+
+</pre>
+
+<p>Deletes are special keys in accumulo that get sorted along will all the other
+data.  When a delete key is inserted, accumulo will not show anything that has
+a timestamp less than or equal to the delete key.  In the example below an
+insert is made with timestamp 5 and then a delete is inserted with timestamp 3.
+The scan after that show that the delete marker does not hide the key.  However
+when a delete is inserted with timestamp 5, then nothing can be seen.  Once a
+delete marker is inserted, it is there until a full major compaction occurs.
+That is why the insert made after the delete can not be seen.  The insert after
+the flush and compact commands can be seen because the delete marker is gone.
+The flush forced a minor compaction and compact forced a full major compaction.
+
+<p><pre>
+root@ac12&gt; createtable bar
+root@ac12 bar&gt; insert -t 5 r1 cf1 cq1 val1
+root@ac12 bar&gt; scan -st
+r1 cf1:cq1 [] 5    val1
+root@ac12 bar&gt; delete -t 3 r1 cf1 cq1     
+root@ac12 bar&gt; scan
+r1 cf1:cq1 []    val1
+root@ac12 bar&gt; scan -st
+r1 cf1:cq1 [] 5    val1
+root@ac12 bar&gt; delete -t 5 r1 cf1 cq1
+root@ac12 bar&gt; scan -st              
+root@ac12 bar&gt; insert -t 5 r1 cf1 cq1 val2
+root@ac12 bar&gt; scan -st
+root@ac12 bar&gt; flush -t bar
+23 14:01:36,587 [shell.Shell] INFO : Flush of table bar initiated...
+root@ac12 bar&gt; compact -t bar
+23 14:02:00,042 [shell.Shell] INFO : Compaction of table bar scheduled for 20100723140200EDT
+root@ac12 bar&gt; insert -t 5 r1 cf1 cq1 val1
+root@ac12 bar&gt; scan
+r1 cf1:cq1 []    val1
+</pre>
+
+<p>If two inserts are made into accumulo with the same row, column, and
+timestamp, then the behavior is non-deterministic.
+
+<p>Accumulo 1.2 introduces the concept of logical time.  This ensures that
+timestamps set by accumulo always move forward.  There have been many problems
+caused by tablet servers with different system times.  In the case where a
+tablet servers time is in the future, tablets hosted on that tablet server and
+then migrated will have future timestamps in their data.  This can cause newer
+keys to fall behind existing keys, which can result in seeing older data or not
+seeing data if a new key falls behind on old delete.  Logical time prevents
+this by ensuring that accumulo set time stamps never go backwards, on a per
+tablet basis.  So if a tablet servers time is a year in the future, then any
+tablet hosted there will generate timestamps a year in the future even when
+later hosted on a server with correct time. Logical time can be configured on a
+per table basis to either set time in millis or to use a per tablet counter.
+The per tablet counter gives unique one up time stamps on a per mutation
+basis. When using time in millis, if two things arrive within the same
+millisecond then both receive the same timestamp. 
+
+<p>The example below shows a table created using a per tablet counter for
+timestamps.  Two inserts are made, the first gets timestamp 0 the second 1.
+After that the table is split into two tablets and two more inserts are made.
+These inserts get the same timestamp because they are made on different
+tablets.    When the original tablet is split into two, the two child tablets
+inherit the next timestamp of their parent and start from there. So do not
+expect this configuration to offer unique timestamps across a table.  Its only
+purpose is to uniquely order events within a tablet.
+
+<p><pre>
+root@ac12 foo&gt; createtable -tl logical
+root@ac12 logical&gt; insert 000892 person name "John Doe"
+root@ac12 logical&gt; insert 003042 person name "Jane Doe"
+root@ac12 logical&gt; scan -st
+000892 person:name [] 0    John Doe
+003042 person:name [] 1    Jane Doe
+root@ac12 logical&gt;
+root@ac12 logical&gt; addsplits -t logical 002000
+root@ac12 logical&gt; insert 003042 person address "123 Somewhere"
+root@ac12 logical&gt; insert 000892 person address "123 Nowhere"  
+root@ac12 logical&gt; scan -st
+000892 person:address [] 2    123 Nowhere
+000892 person:name [] 0    John Doe
+003042 person:address [] 2    123 Somewhere
+003042 person:name [] 1    Jane Doe
+root@ac12 logical&gt; 
+ 
+</pre>
+
+</body>
+</html>


Mime
View raw message