phoenix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jya...@apache.org
Subject svn commit: r1613476 - /phoenix/site/source/src/site/markdown/secondary_indexing.md
Date Fri, 25 Jul 2014 17:02:13 GMT
Author: jyates
Date: Fri Jul 25 17:02:13 2014
New Revision: 1613476

URL: http://svn.apache.org/r1613476
Log:
Adding docs for removal of mutable secondary index deadlock for 0.98.4+

Modified:
    phoenix/site/source/src/site/markdown/secondary_indexing.md

Modified: phoenix/site/source/src/site/markdown/secondary_indexing.md
URL: http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/secondary_indexing.md?rev=1613476&r1=1613475&r2=1613476&view=diff
==============================================================================
--- phoenix/site/source/src/site/markdown/secondary_indexing.md (original)
+++ phoenix/site/source/src/site/markdown/secondary_indexing.md Fri Jul 25 17:02:13 2014
@@ -94,6 +94,7 @@ If the index tables are not setup correc
 Only mutable indexing requires special configuration options in the region server to run
- phoenix ensures that they are setup correctly when you enable mutable indexing on the table;
if the correct properties are not set, you will not be able to turn it on.
 
 You will need to add the following parameters to `hbase-site.xml`:
+
 ```
 <property>
   <name>hbase.regionserver.wal.codec</name>
@@ -103,6 +104,65 @@ You will need to add the following param
 
 This enables custom WAL edits to be written, ensuring proper writing/replay of the index
updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression.
 
+### Advanced Setup - Removing Index Deadlocks (0.98.4+)
+
+Phoenix releases that include these changes (4.1+, 5.0.0+) are still backwards compatible
with older versions of phoenix (to the extent that they are semantically compatible) as well
as with older versions of HBase (0.98.1-0.98.3).
+
+As of HBase 0.98.4 we can finally remove the change of index deadlocks. In HBase you can
tune the number of RPC threads to match client writes + index writes, but there is still a
chance you could have a deadlock in an unlucky scenario (i.e. Client A -> Server A, Client
B -> Server B, each taking the last RPC thread. Then each server attempts to make an index
update to the other, Server A -> Server B, and vice versa, but they can't as there are
no more available RPC threads).
+
+As of [PHOENIX-938](https://issues.apache.org/jira/browse/PHOENIX-938) and [HBASE-11513](https://issues.apache.org/jira/browse/HBASE-11513)
we can remove these deadlocks by providing a different set of RPC handlers for index updates
by giving index updates their own 'rpc priority' and handling the priorities via a custom
Phoenix RPC Handler.
+
+The properties you need to set to enable this are
+
+#### Server Side
+
+```
+<property>
+  <name>hbase.region.server.rpc.scheduler.factory.class</name>
+  <value>org.apache.phoenix.hbase.index.ipc.PhoenixIndexRpcSchedulerFactory</value>
+  <description>Factory to create the Phoenix RPC Scheduler that knows to put index
updates into index queues</description>
+</property>
+```
+
+After adding these settings to your hbase-site.xml, you just need to do a rolling restart
of your cluster.
+
+
+Note that having the configs on both client and server side will not impact correctness or
performance.
+
+#### Tuning
+
+By default, index priority range is between (1000, 1050]. Higher priorites within the index
range, at this time, do not means updates are processed sooner. However, we reserve this range
to provide that possibility in the future.
+
+You can specifiy this range however to suit your individual cluster requirements by adjusting
the follwing parameters
+
+```
+<property>
+	<name>org.apache.phoenix.regionserver.index.priority.min</name>
+	<value>1050</value>
+	<description>Value to specify to bottom (inclusive) of the range in which index priority
may lie</description>
+</property>
+<property>
+	<name>org.apache.phoenix.regionserver.index.priority.max</name>
+	<value>1050</value>
+	<description>Value to specify to top (exclusive) of the range in which index priority
may lie</description>
+</property>
+```
+
+The number of RPC Handler Threads can be specified via:
+
+```
+<property>
+	<name>org.apache.phoenix.regionserver.index.handler.count</name>
+	<value>30</value>
+	<description>Number of threads to use when serving index write requests</description>
+</property>
+```
+
+Though the actual number of threads is dictated by the Max(number of call queues, handler
count), where the number of call queues is determined by standard HBase configuration (see
below).
+
+
+To further tune the queues, you can adjust the standard rpc queue length parameters (currently,
there are no special knobs for the index queues), specifically "ipc.server.max.callqueue.length"
and "ipc.server.callqueue.handler.factor". See the [HBase Reference Guide](http://hbase.apache.org/book.html)
for more details.
+ 
 ## Tuning
 Out the box, indexing is pretty fast. However, to optimize for your particular environment
and workload, there are several properties you can tune.
 



Mime
View raw message