drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject drill-site git commit: updates/edits for drill 1.5
Date Sat, 06 Feb 2016 00:25:14 GMT
Repository: drill-site
Updated Branches:
  refs/heads/asf-site 1ae8531a4 -> 2cb56a575


updates/edits for drill 1.5


Project: http://git-wip-us.apache.org/repos/asf/drill-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill-site/commit/2cb56a57
Tree: http://git-wip-us.apache.org/repos/asf/drill-site/tree/2cb56a57
Diff: http://git-wip-us.apache.org/repos/asf/drill-site/diff/2cb56a57

Branch: refs/heads/asf-site
Commit: 2cb56a5753f3c802fb82584c14ec039cc394a571
Parents: 1ae8531
Author: Bridget Bevens <bbevens@maprtech.com>
Authored: Fri Feb 5 16:25:01 2016 -0800
Committer: Bridget Bevens <bbevens@maprtech.com>
Committed: Fri Feb 5 16:25:01 2016 -0800

----------------------------------------------------------------------
 docs/configuration-options-introduction/index.html           | 4 ++--
 docs/configuring-drill-memory/index.html                     | 6 ++++--
 .../index.html                                               | 6 +++---
 docs/troubleshooting/index.html                              | 8 +++++---
 feed.xml                                                     | 4 ++--
 5 files changed, 16 insertions(+), 12 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/configuration-options-introduction/index.html
----------------------------------------------------------------------
diff --git a/docs/configuration-options-introduction/index.html b/docs/configuration-options-introduction/index.html
index 851655a..34cab78 100644
--- a/docs/configuration-options-introduction/index.html
+++ b/docs/configuration-options-introduction/index.html
@@ -1044,7 +1044,7 @@
 
     </div>
 
-     
+     Feb 6, 2016
 
     <link href="/css/docpage.css" rel="stylesheet" type="text/css">
 
@@ -1293,7 +1293,7 @@ Drill sources the local <code>&lt;drill_installation_directory&gt;/conf</code>
d
 <tr>
 <td>planner.memory.max_query_memory_per_node</td>
 <td>2147483648 bytes</td>
-<td>Sets the maximum estimate of memory for a query per node in bytes. If the estimate
is too low, Drill re-plans the query without memory-constrained operators.</td>
+<td>Sets the maximum amount of direct memory allocated to the sort operator in each
query on a node. If a query plan contains multiple sort operators, they all share this memory.
If you encounter memory issues when running queries with sort operators, increase the value
of this option.</td>
 </tr>
 <tr>
 <td>planner.memory.non_blocking_operators_memory</td>

http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/configuring-drill-memory/index.html
----------------------------------------------------------------------
diff --git a/docs/configuring-drill-memory/index.html b/docs/configuring-drill-memory/index.html
index 64e1a49..59b30b8 100644
--- a/docs/configuring-drill-memory/index.html
+++ b/docs/configuring-drill-memory/index.html
@@ -1042,7 +1042,7 @@
 
     </div>
 
-     
+     Feb 6, 2016
 
     <link href="/css/docpage.css" rel="stylesheet" type="text/css">
 
@@ -1059,7 +1059,9 @@ is written to disk during each phase of a job.</p>
 a Drillbit. The on-heap memory for Drill is typically set at 4-8G (default is 4), which should
 suffice because Drill avoids having data sit in heap memory.</p>
 
-<p>The <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a>
system option value determines the memory limits per node for each running query, especially
for those involving external sorts, such as window functions. When you have a large amount
of direct memory allocated, but still encounter memory issues when running these queries,
increase the value of the option.</p>
+<p>As of Drill 1.5, Drill uses a new allocator that improves an operator’s use of
direct memory and tracks the memory use more accurately. Due to this change, the sort operator
(in queries that ran successfully in previous releases) may not have enough memory, resulting
in a failed query and out of memory error instead of spilling to disk.</p>
+
+<p>The <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a>
system option value sets the maximum amount of direct memory allocated to the sort operator
in each query on a node. If a query plan contains multiple sort operators, they all share
this memory. If you encounter memory issues when running queries with sort operators, increase
the value of this option. If you continue to encounter memory issues after increasing this
value, you can also reduce the value of the <a href="/docs/configuration-options-introduction/"><code>planner.width.max_per_node</code></a>
option to reduce the level of parallelism per node. However, this may increase the amount
of time required for a query to complete.  </p>
 
 <h2 id="modifying-drillbit-memory">Modifying Drillbit Memory</h2>
 

http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
----------------------------------------------------------------------
diff --git a/docs/sort-based-and-hash-based-memory-constrained-operators/index.html b/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
index 246cb6b..7aa1b23 100644
--- a/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
+++ b/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
@@ -1044,7 +1044,7 @@
 
     </div>
 
-     
+     Feb 6, 2016
 
     <link href="/css/docpage.css" rel="stylesheet" type="text/css">
 
@@ -1077,8 +1077,8 @@ Default: 1073741824 Range: 0 - 1073741824</p></li>
 Toggles the state of memory estimation and re-planning of the query. When enabled, Drill
conservatively estimates memory requirements and typically excludes memory-constrained operators
from the plan and negatively impacts performance.<br>
 Default: false</p></li>
 <li><p><strong>planner.memory.max_query_memory_per_node</strong><br>
-Sets the maximum estimate of memory for a query per node. If the estimate is too low, Drill
re-plans the query without memory-constrained operators.<br>
-Default: 2147483648</p></li>
+Sets the maximum amount of direct memory allocated to the sort operator in each query on
a node. If a query plan contains multiple sort operators, they all share this memory. If you
encounter memory issues when running queries with sort operators, increase the value of this
option.<br>
+Default: 2147483648 </p></li>
 </ul>
 
     

http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/troubleshooting/index.html
----------------------------------------------------------------------
diff --git a/docs/troubleshooting/index.html b/docs/troubleshooting/index.html
index 6cc6738..67f1f4e 100644
--- a/docs/troubleshooting/index.html
+++ b/docs/troubleshooting/index.html
@@ -1040,7 +1040,7 @@
 
     </div>
 
-     Jan 5, 2016
+     Feb 6, 2016
 
     <link href="/css/docpage.css" rel="stylesheet" type="text/css">
 
@@ -1106,9 +1106,11 @@
 
 <h3 id="memory-issues">Memory Issues</h3>
 
-<p>Symptom: Memory problems occur when you run certain queries, such as those that
perform window functions.</p>
+<p>Symptom: Memory problems occur when you run certain queries, such as those with
sort operators.</p>
 
-<p>Solution: The <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a>
system option value determines the memory limits per node for each running query, especially
for those involving external sorts, such as window functions. When you have a large amount
of direct memory allocated, but still encounter memory issues when running these queries,
increase the value of the option.</p>
+<p>Solution: Increase the value of the <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a>
option, which sets the maximum amount of direct memory allocated to the sort operator in each
query on a node. If a query plan contains multiple sort operators, they all share this memory.

+If you continue to encounter memory issues after increasing the  <code>planner.memory.max_query_memory_per_node</code>
value, you can also reduce the value of the <code>planner.width.max_per_node</code>
option to reduce the level of parallelism per node. However, this may increase the amount
of time required for a query to complete.
+See <a href="/docs/configuring-drill-memory/">Configuring Drill Memory</a>.</p>
 
 <h3 id="query-parsing-errors">Query Parsing Errors</h3>
 

http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/feed.xml
----------------------------------------------------------------------
diff --git a/feed.xml b/feed.xml
index 0bacba3..f0b787d 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
 </description>
     <link>/</link>
     <atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Tue, 02 Feb 2016 16:01:14 -0800</pubDate>
-    <lastBuildDate>Tue, 02 Feb 2016 16:01:14 -0800</lastBuildDate>
+    <pubDate>Fri, 05 Feb 2016 16:22:11 -0800</pubDate>
+    <lastBuildDate>Fri, 05 Feb 2016 16:22:11 -0800</lastBuildDate>
     <generator>Jekyll v2.5.2</generator>
     
       <item>


Mime
View raw message