drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [13/17] drill git commit: Updates to docs for Drill 1.8
Date Tue, 30 Aug 2016 22:29:28 GMT
Updates to docs for Drill 1.8


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/5465a443
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/5465a443
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/5465a443

Branch: refs/heads/gh-pages
Commit: 5465a443bc50047ac1fc49965dd31a2ab4b2ae1f
Parents: ddb7fcf
Author: Bridget Bevens <bbevens@maprtech.com>
Authored: Sat Aug 13 12:26:05 2016 -0700
Committer: Bridget Bevens <bbevens@maprtech.com>
Committed: Sat Aug 13 12:26:05 2016 -0700

----------------------------------------------------------------------
 .../010-partition-pruning-introduction.md       |  4 +-
 _docs/rn/003-1.8.0-rn.md                        | 47 +++++++++++++++-----
 2 files changed, 38 insertions(+), 13 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/5465a443/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
b/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
index e5f4e5f..f64a633 100644
--- a/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
+++ b/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
@@ -1,12 +1,12 @@
 ---
 title: "Partition Pruning Introduction"
-date: 2016-08-11 19:02:20 UTC
+date: 2016-08-08 18:42:19 UTC
 parent: "Partition Pruning"
 --- 
 
 Partition pruning is a performance optimization that limits the number of files and partitions
that Drill reads when querying file systems and Hive tables. When you partition data, Drill
only reads a subset of the files that reside in a file system or a subset of the partitions
in a Hive table when a query matches certain filter criteria.
 
-As of Drill 1.8, partition pruning also applies to the Parquet metadata cache. When data
is partitioned in a directory hierarchy, Drill attempts to read the metadata cache file from
a sub-partition, based on matching filter criteria instead of reading from the top level partition,
to reduce the amount of metadata read during the query planning time. If you created a metadata
cache file in a previous version of Drill, you must issue the REFRESH TABLE METADATA command
to regenerate the metadata cache file before running queries for partition pruning to occur.
See [Optimizing Parquet Metadata Reading]({{site.baseurl}}/docs/optimizing-parquet-metadata-reading/)
for more information.  
+As of Drill 1.8, partition pruning also applies to the Parquet metadata cache. When data
is partitioned in a directory hierarchy, Drill attempts to read the metadata cache file from
a sub-partition, based on matching filter criteria instead of reading from the top level partition,
to reduce the amount of metadata read during the query planning time. If you created a metadata
cache file in a previous version of Drill, you must issue the REFRESH TABLE METADATA command
to regenerate the metadata cache file before running queries for metadata cache pruning to
occur. See [Optimizing Parquet Metadata Reading]({{site.baseurl}}/docs/optimizing-parquet-metadata-reading/)
for more information.  
 
 The query planner in Drill performs partition pruning by evaluating the filters. If no partition
filters are present, the underlying Scan operator reads all files in all directories and then
sends the data to operators, such as Filter, downstream. When partition filters are present,
the query planner pushes the filters down to the Scan if possible. The Scan reads only the
directories that match the partition filters, thus reducing disk I/O.
 

http://git-wip-us.apache.org/repos/asf/drill/blob/5465a443/_docs/rn/003-1.8.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/003-1.8.0-rn.md b/_docs/rn/003-1.8.0-rn.md
index 65de9a2..78db742 100644
--- a/_docs/rn/003-1.8.0-rn.md
+++ b/_docs/rn/003-1.8.0-rn.md
@@ -3,24 +3,33 @@ title: "Apache Drill 1.8.0 Release Notes"
 parent: "Release Notes"
 ---
 
-**Release date:**  August, 2016
+**Release date:**  August 15, 2016
 
 Today, we're happy to announce the availability of Drill 1.8.0. You can download it [here](https://drill.apache.org/download/).
 
-This release provides metadata cache pruning, support for the IF EXISTS parameter with the
DROP TABLE and DROP VIEW commands, support for the DESCRIBE SCHEMA command, multi-byte delimiter
support, new parameters for filter selectivity estimates, and the following bug fixes and
improvements:  
+This release provides metadata cache pruning, support for the IF EXISTS parameter with the
DROP TABLE and DROP VIEW commands, support for the DESCRIBE SCHEMA command, multi-byte delimiter
support, and new parameters for filter selectivity estimates.  
+
+## Configuration and Launch Script Changes 
+This release of Drill also includes the following changes to the configuration and launch
scripts: 
+
+- Default Drill settings now reside in `$DRILL_HOME/bin/drill-config.sh`. You can override
many settings by creating an entry in `$DRILL_HOME/conf/drill-env.sh`. The file includes descriptions
of the options that you can set.  ([DRILL-4581](https://issues.apache.org/jira/browse/DRILL-4581))
 
+- Due to issues at high concurrency, the native Linux epoll transport is now disabled by
default. ([DRILL-4623](https://issues.apache.org/jira/browse/DRILL-4623))  
+ 
+If you upgrade to Drill 1.8, you must merge your custom settings with the latest settings
in the `drill-override.conf` and `drill-env.sh` file that ships with Drill. As of Drill 1.8,
all Drill defaults reside in the Drill scripts. The `drill-env.sh` script contains only your
customizations. When you merge your existing `drill-env.sh` file with the 1.8 version of the
file, you can remove all of the settings in your file except for those you created yourself.
Consult the original `drill-env.sh` file from the prior Drill release to determine which settings
you can remove.
+
+
+
+Drill 1.8 provides the following bug fixes and improvements:  
 
-    
 <h2>        Sub-task
 </h2>
 <ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4560'>DRILL-4560</a>]
-         ZKClusterCoordinator does not call DrillbitStatusListener.drillbitRegistered for
new bits
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4581'>DRILL-4581</a>]
-         Various problems in the Drill startup scripts
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4728'>DRILL-4728</a>]
-         Add support for new metadata fetch APIs
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4729'>DRILL-4729</a>]
-         Add support for prepared statement implementation on server side
 </li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4730'>DRILL-4730</a>]
-         Update JDBC DatabaseMetaData implementation to use new Metadata APIs
-</li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4732'>DRILL-4732</a>]
-         Update JDBC driver to use the new prepared statement APIs on DrillClient
 </li>
 </ul>
@@ -38,12 +47,16 @@ This release provides metadata cache pruning, support for the IF EXISTS
paramete
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4574'>DRILL-4574</a>]
-         Avro Plugin: Flatten does not work correctly on record items
 </li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4623'>DRILL-4623</a>]
-         Disable Epoll by Default
+</li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4658'>DRILL-4658</a>]
-         cannot specify tab as a fieldDelimiter in table function
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4664'>DRILL-4664</a>]
-         ScanBatch.isNewSchema() returns wrong result for map datatype
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4665'>DRILL-4665</a>]
-         Partition pruning not working for hive partitioned table with &#39;LIKE&#39;
and &#39;=&#39; filter
 </li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4704'>DRILL-4704</a>]
-         select statement behavior is inconsistent for decimal values in parquet
+</li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4707'>DRILL-4707</a>]
-         Conflicting columns names under case-insensitive policy lead to either memory leak
or incorrect result
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4715'>DRILL-4715</a>]
-         Java compilation error for a query with large number of expressions
@@ -68,6 +81,10 @@ This release provides metadata cache pruning, support for the IF EXISTS
paramete
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4825'>DRILL-4825</a>]
-         Wrong data with UNION ALL when querying different sub-directories under the same
table
 </li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4836'>DRILL-4836</a>]
-         ZK Issue during Drillbit startup, possibly due to race condition
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4846'>DRILL-4846</a>]
-         Eliminate extra operations during metadata cache pruning
+</li>
 </ul>
                         
 <h2>        Improvement
@@ -83,11 +100,11 @@ This release provides metadata cache pruning, support for the IF EXISTS
paramete
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4751'>DRILL-4751</a>]
-         Remove dumpcat script from Drill distribution
 </li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4752'>DRILL-4752</a>]
-         Remove submit_plan script from Drill distribution
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4766'>DRILL-4766</a>]
-         FragmentExecutor should use EventProcessor and avoid blocking rpc threads
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4786'>DRILL-4786</a>]
-         Improve metadata cache performance for queries with multiple partitions
 </li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4792'>DRILL-4792</a>]
-         Include session options used for a query as part of the profile
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4822'>DRILL-4822</a>]
-         Extend distrib-env.sh search to consider site directory
 </li>
 </ul>
             
@@ -98,9 +115,17 @@ This release provides metadata cache pruning, support for the IF EXISTS
paramete
 </li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4673'>DRILL-4673</a>]
-         Implement &quot;DROP TABLE IF EXISTS&quot; for drill to prevent FAILED status
on command return
 </li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4714'>DRILL-4714</a>]
-         Add metadata and prepared statement APIs to DrillClient&lt;-&gt;Drillbit
interface
-</li>
 <li>[<a href='https://issues.apache.org/jira/browse/DRILL-4819'>DRILL-4819</a>]
-         Update MapR version to 5.2.0
 </li>
 </ul>
-                                                                   
\ No newline at end of file
+                                                        
+<h2>        Task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4499'>DRILL-4499</a>]
-         Remove unused classes
+</li>
+</ul>
+                  
+
+    
+                                           
\ No newline at end of file


Mime
View raw message