drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tshi...@apache.org
Subject [16/25] drill git commit: html > markdown table format
Date Tue, 12 May 2015 05:56:58 GMT
html > markdown table format


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/598be281
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/598be281
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/598be281

Branch: refs/heads/gh-pages
Commit: 598be2810054cb608cd4854c132f9df6165c6b3c
Parents: 7a40745
Author: Kristine Hahn <khahn@maprtech.com>
Authored: Sat May 9 06:42:11 2015 -0700
Committer: Kristine Hahn <khahn@maprtech.com>
Committed: Sat May 9 06:42:11 2015 -0700

----------------------------------------------------------------------
 .../010-configuration-options-introduction.md   | 349 +++----------------
 1 file changed, 58 insertions(+), 291 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/598be281/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
index 8cfadf8..8443036 100644
--- a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
+++ b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
@@ -14,297 +14,64 @@ The sys.options table in Drill contains information about boot (start-up)
and sy
 ## System Options
 The sys.options table lists the following options that you can set at the session or system
level as described in the section, ["Planning and Execution Options"]({{site.baseurl}}/docs/planning-and-execution-options)

 
-<table>
-  <tr>
-    <th>Name</th>
-    <th>Default</th>
-    <th>Comments</th>
-  </tr>
-  <tr>
-    <td>drill.exec.functions.cast_empty_string_to_null</td>
-    <td>FALSE</td>
-    <td>Not supported in this release.</td>
-  </tr>
-  <tr>
-    <td>drill.exec.storage.file.partition.column.label</td>
-    <td>dir</td>
-    <td>Accepts a string input.</td>
-  </tr>
-  <tr>
-    <td>exec.errors.verbose</td>
-    <td>FALSE</td>
-    <td>Toggles verbose output of executable error messages</td>
-  </tr>
-  <tr>
-    <td>exec.java_compiler</td>
-    <td>DEFAULT</td>
-    <td>Switches between DEFAULT, JDK, and JANINO mode for the current session. Uses
Janino by default for generated source code of less than exec.java_compiler_janino_maxsize;
otherwise, switches to the JDK compiler.</td>
-  </tr>
-  <tr>
-    <td>exec.java_compiler_debug</td>
-    <td>TRUE</td>
-    <td>Toggles the output of debug-level compiler error messages in runtime generated
code.</td>
-  </tr>
-  <tr>
-    <td>exec.java_compiler_janino_maxsize</td>
-    <td>262144</td>
-    <td>See the exec.java_compiler option comment. Accepts inputs of type LONG.</td>
-  </tr>
-  <tr>
-    <td>exec.max_hash_table_size</td>
-    <td>1073741824</td>
-    <td>Ending size for hash tables. Range: 0 - 1073741824</td>
-  </tr>
-  <tr>
-    <td>exec.min_hash_table_size</td>
-    <td>65536</td>
-    <td>Starting size for hash tables. Increase according to available memory to improve
performance. Range: 0 - 1073741824</td>
-  </tr>
-  <tr>
-    <td>exec.queue.enable</td>
-    <td>FALSE</td>
-    <td>Changes the state of query queues to control the number of queries that run
simultaneously.</td>
-  </tr>
-  <tr>
-    <td>exec.queue.large</td>
-    <td>10</td>
-    <td>Sets the number of large queries that can run concurrently in the cluster.
-Range: 0-1000</td>
-  </tr>
-  <tr>
-    <td>exec.queue.small</td>
-    <td>100</td>
-    <td>Sets the number of small queries that can run concurrently in the cluster.
Range: 0-1001</td>
-  </tr>
-  <tr>
-    <td>exec.queue.threshold</td>
-    <td>30000000</td>
-    <td>Sets the cost threshold, which depends on the complexity of the queries in
queue, for determining whether query is large or small. Complex queries have higher thresholds.
Range: 0-9223372036854775807</td>
-  </tr>
-  <tr>
-    <td>exec.queue.timeout_millis</td>
-    <td>300000</td>
-    <td>Indicates how long a query can wait in queue before the query fails. Range:
0-9223372036854775807</td>
-  </tr>
-  <tr>
-    <td>planner.add_producer_consumer</td>
-    <td>FALSE</td>
-    <td>Increase prefetching of data from disk. Disable for in-memory reads.</td>
-  </tr>
-  <tr>
-    <td>planner.affinity_factor</td>
-    <td>1.2</td>
-    <td>Accepts inputs of type DOUBLE.</td>
-  </tr>
-  <tr>
-    <td>planner.broadcast_factor</td>
-    <td>1</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.broadcast_threshold</td>
-    <td>10000000</td>
-    <td>The maximum number of records allowed to be broadcast as part of a query. After
one million records, Drill reshuffles data rather than doing a broadcast to one side of the
join. Range: 0-2147483647</td>
-  </tr>
-  <tr>
-    <td>planner.disable_exchanges</td>
-    <td>FALSE</td>
-    <td>Toggles the state of hashing to a random exchange.</td>
-  </tr>
-  
-  <tr>
-    <td>planner.enable_broadcast_join</td>
-    <td>TRUE</td>
-    <td>Changes the state of aggregation and join operators. Do not disable.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_constant_folding</td>
-    <td>TRUE</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.enable_demux_exchange</td>
-    <td>FALSE</td>
-    <td>Toggles the state of hashing to a demulitplexed exchange.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_hash_single_key</td>
-    <td>TRUE</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.enable_hashagg</td>
-    <td>TRUE</td>
-    <td>Enable hash aggregation; otherwise, Drill does a sort-based aggregation. Does
not write to disk. Enable is recommended.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_hashjoin</td>
-    <td>TRUE</td>
-    <td>Enable the memory hungry hash join. Drill assumes that a query with have adequate
memory to complete and tries to use the fastest operations possible to complete the planned
inner, left, right, or full outer joins using a hash table. Does not write to disk. Disabling
hash join allows Drill to manage arbitrarily large data in a small memory footprint.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_hashjoin_swap</td>
-    <td>TRUE</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.enable_mergejoin</td>
-    <td>TRUE</td>
-    <td>Sort-based operation. A merge join is used for inner join, left and right outer
joins.  Inputs to the merge join must be sorted. It reads the sorted input streams from both
sides and finds matching rows. Writes to disk.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_multiphase_agg</td>
-    <td>TRUE</td>
-    <td>Each minor fragment does a local aggregation in phase 1, distributes on a hash
basis using GROUP-BY keys partially aggregated results to other fragments, and all the fragments
perform a total aggregation using this data.  
- </td>
-  </tr>
-  <tr>
-    <td>planner.enable_mux_exchange</td>
-    <td>TRUE</td>
-    <td>Toggles the state of hashing to a multiplexed exchange.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_nestedloopjoin</td>
-    <td>TRUE</td>
-    <td>Sort-based operation. Writes to disk.</td>
-  </tr>
-  <tr>
-    <td>planner.enable_nljoin_for_scalar_only</td>
-    <td>TRUE</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.enable_streamagg</td>
-    <td>TRUE</td>
-    <td>Sort-based operation. Writes to disk.</td>
-  </tr>
-  <tr>
-    <td>planner.identifier_max_length</td>
-    <td>1024</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.join.hash_join_swap_margin_factor</td>
-    <td>10</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.join.row_count_estimate_factor</td>
-    <td>1</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.memory.average_field_width</td>
-    <td>8</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.memory.enable_memory_estimation</td>
-    <td>FALSE</td>
-    <td>Toggles the state of memory estimation and re-planning of the query.
-When enabled, Drill conservatively estimates memory requirements and typically excludes these
operators from the plan and negatively impacts performance.</td>
-  </tr>
-  <tr>
-    <td>planner.memory.hash_agg_table_factor</td>
-    <td>1.1</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.memory.hash_join_table_factor</td>
-    <td>1.1</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.memory.max_query_memory_per_node</td>
-    <td>2147483648</td>
-    <td>Sets the maximum estimate of memory for a query per node. If the estimate is
too low, Drill re-plans the query without memory-constrained operators.</td>
-  </tr>
-  <tr>
-    <td>planner.memory.non_blocking_operators_memory</td>
-    <td>64</td>
-    <td>Range: 0-2048</td>
-  </tr>
-  <tr>
-    <td>planner.nestedloopjoin_factor</td>
-    <td>100</td>
-    <td></td>
-  </tr>
-    <tr>
-    <td>planner.partitioner_sender_max_threads</td>
-    <td>8</td>
-    <td>Upper limit of threads for outbound queuing.</td>
-  </tr>
-  <tr>
-    <td>planner.partitioner_sender_set_threads</td>
-    <td>-1</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.partitioner_sender_threads_factor</td>
-    <td>1</td>
-    <td></td>
-  </tr>
-  <tr>
-    <td>planner.producer_consumer_queue_size</td>
-    <td>10</td>
-    <td>How much data to prefetch from disk (in record batches) out of band of query
execution</td>
-  </tr>
-  <tr>
-    <td>planner.slice_target</td>
-    <td>100000</td>
-    <td>The number of records manipulated within a fragment before Drill parallelizes
operations.</td>
-  </tr>
-  <tr>
-    <td>planner.width.max_per_node</td>
-    <td>3</td>
-    <td>Maximum number of threads that can run in parallel for a query on a node. A
slice is an individual thread. This number indicates the maximum number of slices per query
for the query’s major fragment on a node.</td>
-  </tr>
-  <tr>
-    <td>planner.width.max_per_query</td>
-    <td>1000</td>
-    <td>Same as max per node but applies to the query as executed by the entire cluster.</td>
-  </tr>
-  <tr>
-    <td>store.format</td>
-    <td>parquet</td>
-    <td>Output format for data written to tables with the CREATE TABLE AS (CTAS) command.
Allowed values are parquet, json, or text. Allowed values: 0, -1, 1000000</td>
-  </tr>
-  <tr>
-    <td>store.json.all_text_mode</a></td>
-    <td>FALSE</td>
-    <td>Drill reads all data from the JSON files as VARCHAR. Prevents schema change
errors.</td>
-  </tr>
-  <tr>
-    <td>store.mongo.all_text_mode</td>
-    <td>FALSE</td>
-    <td>Similar to store.json.all_text_mode for MongoDB.</td>
-  </tr>
-  <tr>
-    <td>store.parquet.block-size</a></td>
-    <td>536870912</td>
-    <td>Sets the size of a Parquet row group to the number of bytes less than or equal
to the block size of MFS, HDFS, or the file system.</td>
-  </tr>
-  <tr>
-    <td>store.parquet.compression</td>
-    <td>snappy</td>
-    <td>Compression type for storing Parquet output. Allowed values: snappy, gzip,
none</td>
-  </tr>
-  <tr>
-    <td>store.parquet.enable_dictionary_encoding*</td>
-    <td>FALSE</td>
-    <td>Do not change.</td>
-  </tr>
-  <tr>
-    <td>store.parquet.use_new_reader</td>
-    <td>FALSE</td>
-    <td>Not supported</td>
-  </tr>
-  <tr>
-    <td>window.enable*</td>
-    <td>FALSE</td>
-    <td>Coming soon.</td>
-  </tr>
-</table>
+| Name                                           | Default    | Comments                
                                                                                         
                                                                                         
                                                                                         
                                                          |
+|------------------------------------------------|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| drill.exec.functions.cast_empty_string_to_null | FALSE      | Not supported in this release.
                                                                                         
                                                                                         
                                                                                         
                                                    |
+| drill.exec.storage.file.partition.column.label | dir        | Accepts a string input. 
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| exec.errors.verbose                            | FALSE      | Toggles verbose output of
executable error messages                                                                
                                                                                         
                                                                                         
                                                         |
+| exec.java_compiler                             | DEFAULT    | Switches between DEFAULT,
JDK, and JANINO mode for the current session. Uses Janino by default for generated source
code of less than exec.java_compiler_janino_maxsize; otherwise, switches to the JDK compiler.
                                                                                         
                                                     |
+| exec.java_compiler_debug                       | TRUE       | Toggles the output of debug-level
compiler error messages in runtime generated code.                                       
                                                                                         
                                                                                         
                                                 |
+| exec.java_compiler_janino_maxsize              | 262144     | See the exec.java_compiler
option comment. Accepts inputs of type LONG.                                             
                                                                                         
                                                                                         
                                                        |
+| exec.max_hash_table_size                       | 1073741824 | Ending size for hash tables.
Range: 0 - 1073741824                                                                    
                                                                                         
                                                                                         
                                                      |
+| exec.min_hash_table_size                       | 65536      | Starting size for hash tables.
Increase according to available memory to improve performance. Range: 0 - 1073741824     
                                                                                         
                                                                                         
                                                    |
+| exec.queue.enable                              | FALSE      | Changes the state of query
queues to control the number of queries that run simultaneously.                         
                                                                                         
                                                                                         
                                                        |
+| exec.queue.large                               | 10         | Sets the number of large
queries that can run concurrently in the cluster. Range: 0-1000                          
                                                                                         
                                                                                         
                                                          |
+| exec.queue.small                               | 100        | Sets the number of small
queries that can run concurrently in the cluster. Range: 0-1001                          
                                                                                         
                                                                                         
                                                          |
+| exec.queue.threshold                           | 30000000   | Sets the cost threshold,
which depends on the complexity of the queries in queue, for determining whether query is
large or small. Complex queries have higher thresholds. Range: 0-9223372036854775807     
                                                                                         
                                                          |
+| exec.queue.timeout_millis                      | 300000     | Indicates how long a query
can wait in queue before the query fails. Range: 0-9223372036854775807                   
                                                                                         
                                                                                         
                                                        |
+| planner.add_producer_consumer                  | FALSE      | Increase prefetching of data
from disk. Disable for in-memory reads.                                                  
                                                                                         
                                                                                         
                                                      |
+| planner.affinity_factor                        | 1.2        | Accepts inputs of type DOUBLE.
                                                                                         
                                                                                         
                                                                                         
                                                    |
+| planner.broadcast_factor                       | 1          |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.broadcast_threshold                    | 10000000   | The maximum number of records
allowed to be broadcast as part of a query. After one million records, Drill reshuffles data
rather than doing a broadcast to one side of the join. Range: 0-2147483647               
                                                                                         
                                                  |
+| planner.disable_exchanges                      | FALSE      | Toggles the state of hashing
to a random exchange.                                                                    
                                                                                         
                                                                                         
                                                      |
+| planner.enable_broadcast_join                  | TRUE       | Changes the state of aggregation
and join operators. Do not disable.                                                      
                                                                                         
                                                                                         
                                                  |
+| planner.enable_constant_folding                | TRUE       |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.enable_demux_exchange                  | FALSE      | Toggles the state of hashing
to a demulitplexed exchange.                                                             
                                                                                         
                                                                                         
                                                      |
+| planner.enable_hash_single_key                 | TRUE       |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.enable_hashagg                         | TRUE       | Enable hash aggregation;
otherwise, Drill does a sort-based aggregation. Does not write to disk. Enable is recommended.
                                                                                         
                                                                                         
                                                     |
+| planner.enable_hashjoin                        | TRUE       | Enable the memory hungry
hash join. Drill assumes that a query with have adequate memory to complete and tries to use
the fastest operations possible to complete the planned inner, left, right, or full outer
joins using a hash table. Does not write to disk. Disabling hash join allows Drill to manage
arbitrarily large data in a small memory footprint. |
+| planner.enable_hashjoin_swap                   | TRUE       |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.enable_mergejoin                       | TRUE       | Sort-based operation. A merge
join is used for inner join, left and right outer joins. Inputs to the merge join must be
sorted. It reads the sorted input streams from both sides and finds matching rows. Writes
to disk.                                                                                 
                                                     |
+| planner.enable_multiphase_agg                  | TRUE       | Each minor fragment does
a local aggregation in phase 1, distributes on a hash basis using GROUP-BY keys partially
aggregated results to other fragments, and all the fragments perform a total aggregation using
this data.                                                                               
                                                     |
+| planner.enable_mux_exchange                    | TRUE       | Toggles the state of hashing
to a multiplexed exchange.                                                               
                                                                                         
                                                                                         
                                                      |
+| planner.enable_nestedloopjoin                  | TRUE       | Sort-based operation. Writes
to disk.                                                                                 
                                                                                         
                                                                                         
                                                      |
+| planner.enable_nljoin_for_scalar_only          | TRUE       |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.enable_streamagg                       | TRUE       | Sort-based operation. Writes
to disk.                                                                                 
                                                                                         
                                                                                         
                                                      |
+| planner.identifier_max_length                  | 1024       |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.join.hash_join_swap_margin_factor      | 10         |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.join.row_count_estimate_factor         | 1          |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.memory.average_field_width             | 8          |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.memory.enable_memory_estimation        | FALSE      | Toggles the state of memory
estimation and re-planning of the query. When enabled, Drill conservatively estimates memory
requirements and typically excludes these operators from the plan and negatively impacts performance.
                                                                                         
                                        |
+| planner.memory.hash_agg_table_factor           | 1.1        |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.memory.hash_join_table_factor          | 1.1        |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.memory.max_query_memory_per_node       | 2147483648 | Sets the maximum estimate
of memory for a query per node. If the estimate is too low, Drill re-plans the query without
memory-constrained operators.                                                            
                                                                                         
                                                      |
+| planner.memory.non_blocking_operators_memory   | 64         | Range: 0-2048           
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.nestedloopjoin_factor                  | 100        |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.partitioner_sender_max_threads         | 8          | Upper limit of threads for
outbound queuing.                                                                        
                                                                                         
                                                                                         
                                                        |
+| planner.partitioner_sender_set_threads         | -1         |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.partitioner_sender_threads_factor      | 1          |                         
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| planner.producer_consumer_queue_size           | 10         | How much data to prefetch
from disk (in record batches) out of band of query execution                             
                                                                                         
                                                                                         
                                                         |
+| planner.slice_target                           | 100000     | The number of records manipulated
within a fragment before Drill parallelizes operations.                                  
                                                                                         
                                                                                         
                                                 |
+| planner.width.max_per_node                     | 3          | Maximum number of threads
that can run in parallel for a query on a node. A slice is an individual thread. This number
indicates the maximum number of slices per query for the query’s major fragment on a node.
                                                                                         
                                                     |
+| planner.width.max_per_query                    | 1000       | Same as max per node but
applies to the query as executed by the entire cluster.                                  
                                                                                         
                                                                                         
                                                          |
+| store.format                                   | parquet    | Output format for data written
to tables with the CREATE TABLE AS (CTAS) command. Allowed values are parquet, json, or text.
Allowed values: 0, -1, 1000000                                                           
                                                                                         
                                                |
+| store.json.all_text_mode                       | FALSE      | Drill reads all data from
the JSON files as VARCHAR. Prevents schema change errors.                                
                                                                                         
                                                                                         
                                                         |
+| store.mongo.all_text_mode                      | FALSE      | Similar to store.json.all_text_mode
for MongoDB.                                                                             
                                                                                         
                                                                                         
                                               |
+| store.parquet.block-size                       | 536870912  | Sets the size of a Parquet
row group to the number of bytes less than or equal to the block size of MFS, HDFS, or the
file system.                                                                             
                                                                                         
                                                       |
+| store.parquet.compression                      | snappy     | Compression type for storing
Parquet output. Allowed values: snappy, gzip, none                                       
                                                                                         
                                                                                         
                                                      |
+| store.parquet.enable_dictionary_encoding*      | FALSE      | Do not change.          
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| store.parquet.use_new_reader                   | FALSE      | Not supported           
                                                                                         
                                                                                         
                                                                                         
                                                          |
+| window.enable*                                 | FALSE      | Coming soon.            
                                                                                         
                                                                                         
                                                                                         
                                                          |
 
 \* Not supported in this release.
 


Mime
View raw message