Configuration and Tuning

@@ -235,15 +236,6 @@

License

- Sponsorship -

- Thanks -

- Security -

@@ -294,34 +286,28 @@ Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading -

- Query Server +

- Tracing + Secondary Indexes

- ARRAY type + Statistics Collection

- Sequences + Row Timestamp Column

- Statistics Collection + Salted Tables

- Joins + Skip Scan

- Subqueries +

Views @@ -330,19 +316,19 @@ Multi tenancy

- Paged Queries + Dynamic Columns

- Dynamic Columns +

- Skip Scan + Bulk Loading

- Salted Tables + Query Server

- Row Timestamp Column + Tracing

Metrics @@ -361,6 +347,24 @@

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/subqueries.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/subqueries.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/subqueries.html (original) +++ phoenix/site/publish/subqueries.html Fri Mar 11 01:34:29 2016 @@ -1,7 +1,7 @@ @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@

@@ -373,15 +374,6 @@ ORDER BY count(*) DESC;

License

- Sponsorship -

- Thanks -

- Security -

@@ -432,34 +424,28 @@ ORDER BY count(*) DESC; Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading +

- Query Server + Secondary Indexes

- Tracing + Statistics Collection

- ARRAY type + Row Timestamp Column

- Sequences + Salted Tables

- Statistics Collection + Skip Scan

- Joins -

- Subqueries +

Views @@ -468,19 +454,19 @@ ORDER BY count(*) DESC; Multi tenancy

- Paged Queries + Dynamic Columns

- Dynamic Columns +

- Skip Scan + Bulk Loading

- Salted Tables + Query Server

- Row Timestamp Column + Tracing

Metrics @@ -499,6 +485,24 @@ ORDER BY count(*) DESC;

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/team.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/team.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/team.html (original) +++ phoenix/site/publish/team.html Fri Mar 11 01:34:29 2016 @@ -1,7 +1,7 @@ @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@

@@ -386,15 +387,6 @@

License

- Sponsorship -

- Thanks -

- Security -

@@ -445,34 +437,28 @@ Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading -

- Query Server +

- Tracing + Secondary Indexes

- ARRAY type + Statistics Collection

- Sequences + Row Timestamp Column

- Statistics Collection + Salted Tables

- Joins + Skip Scan

- Subqueries +

Views @@ -481,19 +467,19 @@ Multi tenancy

- Paged Queries + Dynamic Columns

- Dynamic Columns +

- Skip Scan + Bulk Loading

- Salted Tables + Query Server

- Row Timestamp Column + Tracing

Metrics @@ -512,6 +498,24 @@

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/tracing.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/tracing.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/tracing.html (original) +++ phoenix/site/publish/tracing.html Fri Mar 11 01:34:29 2016 @@ -1,7 +1,7 @@ @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@

@@ -411,15 +412,6 @@ Connection conn = DriverManager.getConne

License

- Sponsorship -

- Thanks -

- Security -

@@ -470,34 +462,28 @@ Connection conn = DriverManager.getConne Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading +

- Query Server -

- Tracing + Secondary Indexes

- ARRAY type + Statistics Collection

- Sequences + Row Timestamp Column

- Statistics Collection + Salted Tables

- Joins + Skip Scan

- Subqueries +

Views @@ -506,19 +492,19 @@ Connection conn = DriverManager.getConne Multi tenancy

- Paged Queries -

Dynamic Columns

- Skip Scan +

- Salted Tables + Bulk Loading

- Row Timestamp Column + Query Server +

+ Tracing

Metrics @@ -537,6 +523,24 @@ Connection conn = DriverManager.getConne

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/transactions.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/transactions.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/transactions.html (original) +++ phoenix/site/publish/transactions.html Fri Mar 11 01:34:29 2016 @@ -1,7 +1,7 @@ @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@

@@ -308,15 +309,6 @@ DELETE FROM my_other_table WHERE k=2;

License

- Sponsorship -

- Thanks -

- Security -

@@ -367,34 +359,28 @@ DELETE FROM my_other_table WHERE k=2; Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading -

- Query Server +

- Tracing + Secondary Indexes

- ARRAY type + Statistics Collection

- Sequences + Row Timestamp Column

- Statistics Collection + Salted Tables

- Joins + Skip Scan

- Subqueries +

Views @@ -403,19 +389,19 @@ DELETE FROM my_other_table WHERE k=2; Multi tenancy

- Paged Queries + Dynamic Columns

- Dynamic Columns +

- Skip Scan + Bulk Loading

- Salted Tables + Query Server

- Row Timestamp Column + Tracing

Metrics @@ -434,6 +420,24 @@ DELETE FROM my_other_table WHERE k=2;

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/tuning.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/tuning.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/tuning.html (original) +++ phoenix/site/publish/tuning.html Fri Mar 11 01:34:29 2016 @@ -1,7 +1,7 @@ @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@

@@ -144,7 +145,7 @@ -

Phoenix provides many different knobs and dials to configure and tune the system to run more optimally on your cluster. The configuration is done through a series of Phoenix-specific properties specified both on client and server-side hbase-site.xml files. In addition to these properties, there are of course all the HBase configuration properties with the most important ones documented here.

The table below outlines the full set of Phoenix-specific configuration properties and their defaults. Of these, weâll talk in depth about some of the most important ones below.

@@ -153,13 +154,23 @@ + + + + + + + + + + - + - + @@ -174,27 +185,27 @@ - + - + - + - + - + @@ -264,9 +275,29 @@ - + + + + + + + + + + + + + + + + + + + + + @@ -372,53 +403,9 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Default
data.tx.snapshot.dir	Server-side property specifying the HDFS directory used to store snapshots of the transaction state. No default value.	None
data.tx.timeout	Server-side property specifying the timeout in seconds for a transaction to complete. Default is 30 seconds.	30
phoenix.query.timeoutMs	Number of milliseconds after which a query will timeout on the client. Default is 10 min.	Client-side property specifying the number of milliseconds after which a query will timeout on the client. Default is 10 min.	600000
phoenix.query.keepAliveMs	When the number of threads is greater than the core in the client side thread pool executor, this is the maximum time in milliseconds that excess idle threads will wait for a new tasks before terminating. Default is 60 sec.	Maximum time in milliseconds that excess idle threads will wait for a new tasks before terminating when the number of threads is greater than the cores in the client side thread pool executor. Default is 60 sec.	60000
phoenix.stats.guidepost.width	A server-side parameter that specifies the number of bytes between guideposts. A smaller amount increases parallelization, but also increases the number of chunks which must be merged on the client side. The default value is 100 MB.	Server-side parameter that specifies the number of bytes between guideposts. A smaller amount increases parallelization, but also increases the number of chunks which must be merged on the client side. The default value is 100 MB.	104857600
phoenix.stats.guidepost.per.region	A server-side parameter that specifies the number of guideposts per region. If set to a value greater than zero, then the guidepost width is determiend by `MAX_FILE_SIZE of table / phoenix.stats.guidepost.per.region`. Otherwise, if not set, then the `phoenix.stats.guidepost.width` parameter is used. No default value.	Server-side parameter that specifies the number of guideposts per region. If set to a value greater than zero, then the guidepost width is determiend by `MAX_FILE_SIZE of table / phoenix.stats.guidepost.per.region`. Otherwise, if not set, then the `phoenix.stats.guidepost.width` parameter is used. No default value.	None
phoenix.stats.updateFrequency	A server-side paramater that determines the frequency in milliseconds for which statistics will be refreshed from the statistics table and subsequently used by the client. The default value is 15 min.	Server-side paramater that determines the frequency in milliseconds for which statistics will be refreshed from the statistics table and subsequently used by the client. The default value is 15 min.	900000
phoenix.stats.minUpdateFrequency	A client-side parameter that determines the minimum amount of time in milliseconds that must pass before statistics may again be manually collected through another `UPDATE STATISTICS` call. The default value is `phoenix.stats.updateFrequency / 2`.	Client-side parameter that determines the minimum amount of time in milliseconds that must pass before statistics may again be manually collected through another `UPDATE STATISTICS` call. The default value is `phoenix.stats.updateFrequency / 2`.	450000
phoenix.stats.useCurrentTime	An advanced server-side parameter that if true causes the current time on the server-side to be used as the timestamp of rows in the statistics table when background tasks such as compactions or splits occur. If false, then the max timestamp found while traversing the table over which statistics are being collected is used as the timestamp. Unless your client is controlling the timestamps while reading and writing data, this parameter should be left alone. The default value is true.	Server-side parameter that if true causes the current time on the server-side to be used as the timestamp of rows in the statistics table when background tasks such as compactions or splits occur. If false, then the max timestamp found while traversing the table over which statistics are being collected is used as the timestamp. Unless your client is controlling the timestamps while reading and writing data, this parameter should be left alone. The default value is true.	true
phoenix.query.useIndexes	Determines whether or not indexes are considered by the optimizer to satisfy a query. Default is true	Client-side property determining whether or not indexes are considered by the optimizer to satisfy a query. Default is true	true
phoenix.index.failure.handling.rebuild	Server-side property determining whether or not a mutable index is rebuilt in the background in the event of a commit failure. Only applicable for indexes on mutable, non transactional tables. Default is true	true
phoenix.index.failure.block.write	Server-side property determining whether or not a writes to the data table are disallowed in the event of a commit failure until the index can be caught up with the data table. Requires that phoenix.index.failure.handling.rebuild is true as well. Only applicable for indexes on mutable, non transactional tables. Default is false	false
phoenix.index.failure.handling.rebuild.interval	Server-side property controlling the millisecond frequency at which the server checks whether or not a mutable index needs to be partially rebuilt to catch up with updates to the data table. Only applicable for indexes on mutable, non transactional tables. Default is 10 seconds.	10000
phoenix.index.failure.handling.rebuild.overlap.time	Server-side property controlling how many milliseconds to go back from the timestamp at which the failure occurred to go back when a partial rebuild is performed. Only applicable for indexes on mutable, non transactional tables. Default is 1 millisecond.	1
phoenix.index.mutableBatchSizeThreshold	Number of mutations in a batch beyond which index metadata will be sent as a separate RPC to each region server as opposed to included inline with each mutation. Defaults to 5.	Determines whether or not transactions are enabled in Phoenix. A table may not be declared as transactional if transactions are disabled. Default is false. This is a client side parameter. Available starting from Phoenix 4.7.	false
data.tx.snapshot.dir	The HDFS directory used to store snapshots of the transaction state. No default value. This is a server side parameter. Available starting from Phoenix 4.7.	None
data.tx.timeout	The timeout in seconds for a transaction to complete. Default is 30 seconds. This is a server side parameter. Available starting from Phoenix 4.7.	30
~~phoenix.query.targetConcurrency~~ Obsolete as of 3.2/4.2	Target concurrent threads to use for a query. It serves as a soft limit on the number of scans into which a query may be split. The value should not exceed the hard limit imposed by `phoenix.query.maxConcurrency`.	32
~~phoenix.query.maxConcurrency~~ Obsolete as of 3.2/4.2	Maximum concurrent threads to use for a query. It servers as a hard limit on the number of scans into which a query may be split. A soft limit is imposed by `phoenix.query.targetConcurrency`.	64
~~phoenix.query.maxStatsAge~~ Obsolete as of 3.2/4.2	The maximum age of stats in milliseconds after which they will no longer be used (i.e. the stats were not able to be updated in this amount of time and thus are considered too old). Default is 1 day.	1
~~phoenix.query.statsUpdateFrequency~~ Obsolete as of 3.2/4.2	The frequency in milliseconds at which the stats for each table will be updated. Default is 15 min.	900000
~~phoenix.query.maxIntraRegionParallelization~~ Obsolete as of 3.2/4.2	The maximum number of threads that will be spawned to process data within a single region during query execution	64

-
-

Parallelization

Phoenix breaks up queries into multiple scans and runs them in parallel through coprocessors to improve performance. Hari Kumar, from Ericsson Labs, did a good job of explaining the performance benefits of parallelization and coprocessors here.

As of 3.2/4.2, parallelization in Phoenix is driven by the guideposts as determined by the configuration parameters for statistics collection. Each chunk of data between guideposts will be run in parallel in a separate scan to improve query performance. The chunk size is determined by the server-side phoenix.stats.guidepost.width or phoenix.stats.guidepost.per.region configuration parameters. As the size of the chunks decrease, you'll want to increase phoenix.query.queueSize as more work will be queued in that case. Note that at a minimum, separate scans will be run for each table region. Beyond the statistics collection configuration parameters, the client-side phoenix.query.threadPoolSize and phoenix.query.queueSize parameters and the server-side hbase.regionserver.handler.count parameter have an impact on performance.

@@ -491,15 +478,6 @@

License

- Sponsorship -

- Thanks -

- Security -

@@ -550,34 +528,28 @@ Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading -

- Query Server +

- Tracing + Secondary Indexes

- ARRAY type + Statistics Collection

- Sequences + Row Timestamp Column

- Statistics Collection + Salted Tables

- Joins + Skip Scan

- Subqueries +

Views @@ -586,19 +558,19 @@ Multi tenancy

- Paged Queries + Dynamic Columns

- Dynamic Columns +

- Skip Scan + Bulk Loading

- Salted Tables + Query Server

- Row Timestamp Column + Tracing

Metrics @@ -617,6 +589,24 @@

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/udf.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/udf.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/udf.html (original) +++ phoenix/site/publish/udf.html Fri Mar 11 01:34:29 2016 @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@

@@ -361,15 +362,6 @@ Connection conn = DriverManager.getConne

License

- Sponsorship -

- Thanks -

- Security -

@@ -419,35 +411,29 @@ Connection conn = DriverManager.getConne

Transactions

- Secondary Indexes -

User-defined Functions

- Bulk Loading -

- Query Server +

- Tracing + Secondary Indexes

- ARRAY type + Statistics Collection

- Sequences + Row Timestamp Column

- Statistics Collection + Salted Tables

- Joins + Skip Scan

- Subqueries +

Views @@ -456,19 +442,19 @@ Connection conn = DriverManager.getConne Multi tenancy

- Paged Queries + Dynamic Columns

- Dynamic Columns +

- Skip Scan + Bulk Loading

- Salted Tables + Query Server

- Row Timestamp Column + Tracing

Metrics @@ -487,6 +473,24 @@ Connection conn = DriverManager.getConne

Datatypes

+ ARRAY type +

+ +

+ Row Value Constructors +

+ Sequences +

+ Joins +

+ Subqueries +

Modified: phoenix/site/publish/update_statistics.html URL: http://svn.apache.org/viewvc/phoenix/site/publish/update_statistics.html?rev=1734488&r1=1734487&r2=1734488&view=diff ============================================================================== --- phoenix/site/publish/update_statistics.html (original) +++ phoenix/site/publish/update_statistics.html Fri Mar 11 01:34:29 2016 @@ -69,9 +69,6 @@

@@ -121,6 +116,12 @@