drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [2/2] drill git commit: DRILL-2908
Date Thu, 01 Oct 2015 01:04:09 GMT
DRILL-2908

DRILL-2908 edits

use hbase to lowercase

DRILL-3799

minor edit

add available in Drill 1.2

DRILL-3209

DRILL-3209

minor tweak

minor edit

Changes to https support

more changes to https support

final doc changes for HTTPS support change

formatting

typo

boot option to start-up

add plugin ss

indent image

more indentation fix

add note about image

note format

revise step 5


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/0cfc6b62
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/0cfc6b62
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/0cfc6b62

Branch: refs/heads/gh-pages
Commit: 0cfc6b620da42a97db3a8707e4a51bb8baf4087e
Parents: 297b430
Author: Kristine Hahn <khahn@maprtech.com>
Authored: Tue Sep 29 13:25:33 2015 -0700
Committer: Kristine Hahn <khahn@maprtech.com>
Committed: Wed Sep 30 17:18:44 2015 -0700

----------------------------------------------------------------------
 .../010-configure-drill-introduction.md         |   2 +-
 ...-configuring-web-ui-and-rest-api-security.md |   8 +++--
 .../010-configuration-options-introduction.md   |   1 +
 .../020-start-up-options.md                     |   2 ++
 .../020-storage-plugin-registration.md          |   2 +-
 .../035-plugin-configuration-basics.md          |  19 ++++++++----
 .../040-parquet-format.md                       |  29 ++++++++++++-------
 _docs/img/storage_plugin_config.png             | Bin 0 -> 52174 bytes
 _docs/install/060-starting-the-web-ui.md        |  25 ++++++++++++++--
 _docs/query-data/030-querying-hbase.md          |   8 ++---
 _docs/query-data/050-querying-hive.md           |   7 ++++-
 .../data-types/010-supported-data-types.md      |   9 ++++--
 .../sql-functions/020-data-type-conversion.md   |  10 +++----
 13 files changed, 87 insertions(+), 35 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/configure-drill/010-configure-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/010-configure-drill-introduction.md b/_docs/configure-drill/010-configure-drill-introduction.md
index 6b9a3fa..d338015 100644
--- a/_docs/configure-drill/010-configure-drill-introduction.md
+++ b/_docs/configure-drill/010-configure-drill-introduction.md
@@ -25,4 +25,4 @@ statements is Parquet. Using a configuration option, you can modify Drill
to sto
 
 ## Query Profile Data Storage Configuration
 
-To enjoy a problem-free Drill Web Console experience, you need to [configure the ZooKeeper
PStore]({{site.baseurl}}/docs/persistent-configuration-storage/#configuring-zookeeper-pstore).
+To avoid problems working with the Web Console, you need to [configure the ZooKeeper PStore]({{site.baseurl}}/docs/persistent-configuration-storage/#configuring-zookeeper-pstore).

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/configure-drill/078-configuring-web-ui-and-rest-api-security.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/078-configuring-web-ui-and-rest-api-security.md b/_docs/configure-drill/078-configuring-web-ui-and-rest-api-security.md
index 9068090..9816a30 100644
--- a/_docs/configure-drill/078-configuring-web-ui-and-rest-api-security.md
+++ b/_docs/configure-drill/078-configuring-web-ui-and-rest-api-security.md
@@ -8,9 +8,13 @@ you can limit the access of certain users to Web Console functionality, such
as
 With Web Console security in place, users who do not have administrator privileges need to
use the SHOW SCHEMAS command instead of the Web Console for storage plugin configuration information.
 
 ## HTTPS Support
-Drill 1.2 uses the Linux Pluggable Authentication Module (PAM) and code-level support for
transport layer security (TLS) to secure the Web Console and REST API. By default, the Web
Console and REST API now support the HTTPS protocol.
+Drill 1.2 uses the Linux Pluggable Authentication Module (PAM) and code-level support for
transport layer security (TLS) to secure the Web Console and REST API. By default, the Web
Console and REST API support the HTTP protocol. You set the following start-up option to TRUE
to enable HTTPS support:
 
-By default, Drill generates a self-signed certificate that works with SSL for HTTPS access
to the Web Console. Because Drill uses a self-signed certificate, you see a warning in the
browser when you go to `https://<node IP address>:8047`. The Chrome browser, for example,
requires you to click `Advanced`, and then `Proceed to <address> (unsafe)`.  If you
have a signed certificate by an authority, you can set up a custom SSL to avoid this warning.
You can set up SSL to specify the keystore or truststore, or both, for your organization,
as described in the next section.
+`drill.exec.http.ssl_enabled`
+
+By default this start-up option is set to FALSE.
+
+Drill generates a self-signed certificate that works with SSL for HTTPS access to the Web
Console. Because Drill uses a self-signed certificate, you see a warning in the browser when
you go to `https://<node IP address>:8047`. The Chrome browser, for example, requires
you to click `Advanced`, and then `Proceed to <address> (unsafe)`. If you have a signed
certificate by an authority, you can set up a custom SSL to avoid this warning. You can set
up SSL to specify the keystore or truststore, or both, for your organization, as described
in the next section.
 
 ## Setting Up a Custom SSL Configuration
 

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
index d541a74..6aa9017 100644
--- a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
+++ b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
@@ -71,6 +71,7 @@ The sys.options table lists the following options that you can set as a
system o
 | planner.width.max_per_node                     | 3                | Maximum number of threads
that can run in parallel for a query on a node. A slice is an individual thread. This number
indicates the maximum number of slices per query for the query’s major fragment on a node.
                                                                                         
                                                     |
 | planner.width.max_per_query                    | 1000             | Same as max per node
but applies to the query as executed by the entire cluster. For example, this value might
be the number of active Drillbits, or a higher number to return results faster.          
                                                                                         
                                                              |
 | store.format                                   | parquet          | Output format for data
written to tables with the CREATE TABLE AS (CTAS) command. Allowed values are parquet, json,
psv, csv, or tsv.                                                                        
                                                                                         
                                                         |
+| store.hive.optimize_scan_with_native_readers   | FALSE            | Optimize reads of Parquet-backed
external tables from Hive by using Drill native readers instead of the Hive Serde interface.
(Drill 1.2 and later)                                                                    
                                                                                         
                                               |
 | store.json.all_text_mode                       | FALSE            | Drill reads all data
from the JSON files as VARCHAR. Prevents schema change errors.                           
                                                                                         
                                                                                         
                                                              |
 | store.json.extended_types                      | FALSE            | Turns on special JSON
structures that Drill serializes for storing more type information than the [four basic JSON
types](http://docs.mongodb.org/manual/reference/mongodb-extended-json/).                 
                                                                                         
                                                          |
 | store.json.read_numbers_as_double              | FALSE            | Reads numbers with
or without a decimal point as DOUBLE. Prevents schema change errors.                     
                                                                                         
                                                                                         
                                                                |

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/configure-drill/configuration-options/020-start-up-options.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/configuration-options/020-start-up-options.md b/_docs/configure-drill/configuration-options/020-start-up-options.md
index 1f981fc..c925f0b 100644
--- a/_docs/configure-drill/configuration-options/020-start-up-options.md
+++ b/_docs/configure-drill/configuration-options/020-start-up-options.md
@@ -45,6 +45,8 @@ override.conf` file located in Drill’s` /conf` directory.
 
 The summary of start-up options, also known as boot options, lists default values. The following
descriptions provide more detail on key options that are frequently reconfigured:
 
+* drill.exec.http.ssl_enabled  
+  Available in Drill 1.2. Enables or disables [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/#https-support).
Settings are TRUE and FALSE, respectively. The default is FALSE.  
 * drill.exec.sys.store.provider.class  
   Defines the persistent storage (PStore) provider. The [PStore]({{ site.baseurl }}/docs/persistent-configuration-storage)
holds configuration and profile data.  
 * drill.exec.buffer.size  

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/connect-a-data-source/020-storage-plugin-registration.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/020-storage-plugin-registration.md b/_docs/connect-a-data-source/020-storage-plugin-registration.md
index 11676ad..e2d19c3 100644
--- a/_docs/connect-a-data-source/020-storage-plugin-registration.md
+++ b/_docs/connect-a-data-source/020-storage-plugin-registration.md
@@ -2,7 +2,7 @@
 title: "Storage Plugin Registration"
 parent: "Connect a Data Source"
 ---
-You connect Drill to a file system, Hive, HBase, or other data source through a storage plugin.
On the Storage tab of the Drill Web Console, you can view and reconfigure a storage plugin
if you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/)
to do so. Go to `https://<IP address>:8047/storage`, where IP address is the host name
or IP address of one of the installed Drillbits in a distributed system or `localhost` in
an embedded system. In Drill 1.1 and earlier, go to `http://<IP address>:8047/storage`
to view and configure a storage plugin.
+You connect Drill to a file system, Hive, HBase, or other data source through a storage plugin.
On the Storage tab of the Drill Web Console, you can view and reconfigure a storage plugin.
If [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/#https-support)
is not enabled (the default), go to `http://<IP address>:8047/storage` to view and configure
a storage plugin. IP address is the host name or IP address of one of the installed Drillbits
in a distributed system or `localhost` in an embedded system. If HTTPS support is enabled
and you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/)
to view and configure a storage plugin, go to `https://<IP address>:8047/storage`. 
 
 The Drill installation registers the `cp`, `dfs`, `hbase`, `hive`, and `mongo` default storage
plugin configurations.
 

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/connect-a-data-source/035-plugin-configuration-basics.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/035-plugin-configuration-basics.md b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
index b88ce58..12cbe26 100644
--- a/_docs/connect-a-data-source/035-plugin-configuration-basics.md
+++ b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
@@ -9,16 +9,23 @@ restart any of the Drillbits when you add or update a storage plugin configurati
 
 ## Using the Drill Web Console
 
-Assuming you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/),
you can use the Drill Web Console to update or add a new storage plugin configuration. The
Drill shell needs to be running to access the Drill Web Console. In Drill 1.2 and later, to
open the Drill Web Console, launch a web browser, and go to: `https://<IP address or host
name>:8047` of any Drillbit in the cluster. In Drill 1.1 and earlier, use `http` instead
of `https`. Select the Storage tab to view, update, or add a new storage plugin configuration.

+You can use the Drill Web Console to update or add a new storage plugin configuration. The
Drill shell needs to be running to start the Web Console. 
 
 To create a name and new configuration:
 
-1. Enter a name in **New Storage Plugin**.
+1. [Start the Drill shell]({{site.baseurl}}/docs/starting-drill-on-linux-and-mac-os-x/).
 
+2. [Start the Web Console]({{site.baseurl}}/docs/starting-the-web-console/).  
+   The storage tab appears on the Web Console if you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/)
to view, update, or add storage plugins.  
+3. On the Storage tab, enter a name in **New Storage Plugin**.
    Each configuration registered with Drill must have a distinct
-name. Names are case-sensitive.
-2. Click **Create**.  
-3. In Configuration, it is recommended that you modify a copy of an existing configuration
if possible. Reconfigure attributes of the storage plugin using JSON formatting. The Storage
Plugin Attributes table in the next section describes attributes typically reconfigured by
users. 
-4. Click **Create**.
+name. Names are case-sensitive.  
+     ![sandbox plugin]({{ site.baseurl }}/docs/img/storage_plugin_config.png)
+
+    {% include startnote.html %}The URL differs depending on your installation and configuration.{%
include endnote.html %}  
+4. Click **Create**.  
+5. In Configuration, use JSON formatting to modify a copy of an existing configuration if
possible.  
+   Using a copy of an existing configuration reduces the risk of JSON coding errors. Use
the Storage Plugin Attributes table in the next section as a guide for making typical modifications.
 
+6. Click **Create**.
 
 ## Storage Plugin Attributes
 The following graphic shows key attributes of a typical `dfs`-based storage plugin configuration:
 

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/data-sources-and-file-formats/040-parquet-format.md
----------------------------------------------------------------------
diff --git a/_docs/data-sources-and-file-formats/040-parquet-format.md b/_docs/data-sources-and-file-formats/040-parquet-format.md
index a2f48a6..4d1ca3f 100644
--- a/_docs/data-sources-and-file-formats/040-parquet-format.md
+++ b/_docs/data-sources-and-file-formats/040-parquet-format.md
@@ -48,25 +48,25 @@ To read or write Parquet data, you need to include the Parquet format
in the sto
 
 Use the `store.format` option to set the CTAS output format of a Parquet row group at the
session or system level.
 
-Use the ALTER command to set the `store.format` option.
-         
-        ALTER SESSION SET `store.format` = 'parquet';
-        ALTER SYSTEM SET `store.format` = 'parquet';
+Use the ALTER command to set the `store.format` option.  
+
+``ALTER SESSION SET `store.format` = 'parquet';``  
+``ALTER SYSTEM SET `store.format` = 'parquet';``  
         
 ### Configuring the Size of Parquet Files
 Configuring the size of Parquet files by setting the `store.parquet.block-size` can improve
write performance. The block size is the size of MFS, HDFS, or the file system. 
 
 The larger the block size, the more memory Drill needs for buffering data. Parquet files
that contain a single block maximize the amount of data Drill stores contiguously on disk.
Given a single row group per file, Drill stores the entire Parquet file onto the block, avoiding
network I/O.
 
-To maximize performance, set the target size of a Parquet row group to the number of bytes
less than or equal to the block size of MFS, HDFS, or the file system by using the `store.parquet.block-size`:
        
-        
-        ALTER SESSION SET `store.parquet.block-size` = 536870912;         
-        ALTER SYSTEM SET `store.parquet.block-size` = 536870912  
+To maximize performance, set the target size of a Parquet row group to the number of bytes
less than or equal to the block size of MFS, HDFS, or the file system by using the `store.parquet.block-size`:
 
+
+``ALTER SESSION SET `store.parquet.block-size` = 536870912;``  
+``ALTER SYSTEM SET `store.parquet.block-size` = 536870912``  
 
 The default block size is 536870912 bytes.
 
 ### Type Mapping
-The high correlation between Parquet and SQL data types makes reading Parquet files effortless
in Drill. Writing to Parquet files takes more work than reading. Because SQL does not support
all Parquet data types, to prevent Drill from inferring a type other than one you want, use
the [cast function] ({{ site.baseurl }}/docs/data-type-conversion/#cast) Drill offers more
liberal casting capabilities than SQL for Parquet conversions if the Parquet data is of a
logical type. 
+The high correlation between Parquet and SQL data types makes reading Parquet files effortless
in Drill. Writing to Parquet files takes more work than reading. Because SQL does not support
all Parquet data types, to prevent Drill from inferring a type other than one you want, use
the [cast function]({{ site.baseurl }}/docs/data-type-conversion/#cast) Drill offers more
liberal casting capabilities than SQL for Parquet conversions if the Parquet data is of a
logical type. 
 
 The following general process converts a file from JSON to Parquet:
 
@@ -145,7 +145,16 @@ The first table in this section maps SQL data types to Parquet data types,
limit
 | FLOAT    | FLOAT        | 4-byte single precision floating point number |
 | DOUBLE   | DOUBLE       | 8-byte double precision floating point number |
 | INTEGER  | INT32        | 4-byte signed integer                         |
-| None     | INT96        | 12-byte signed int                            |
+| None*    | INT96        | 12-byte signed int                            |
+
+\* Drill 1.2 and later supports reading the Parquet INT96 type.
+
+## About INT96 Support
+Drill 1.2 and later supports reading the Parquet INT96 type. For example, to decode a timestamp
from Hive or Impala, which is of type INT96, use the CONVERT_FROM function and the [TIMESTAMP_IMPALA]({{site.baseurl}}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
type argument:
+
+``SELECT CONVERT_FROM(timestamp_field, 'TIMESTAMP_IMPALA') as timestamp_field FROM `dfs.file_with_timestamp.parquet`;``
+
+Because INT96 is supported for reads only, you cannot use the TIMESTAMP_IMPALA as a data
type argument with CONVERT_TO.
 
 ### SQL Types to Parquet Logical Types
 Parquet also supports logical types, fully described on the [Apache Parquet site](https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md).
Embedded types, JSON and BSON, annotate a binary primitive type representing a JSON or BSON
document. The logical types and their mapping to SQL types are:

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/img/storage_plugin_config.png
----------------------------------------------------------------------
diff --git a/_docs/img/storage_plugin_config.png b/_docs/img/storage_plugin_config.png
new file mode 100644
index 0000000..966dc98
Binary files /dev/null and b/_docs/img/storage_plugin_config.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/install/060-starting-the-web-ui.md
----------------------------------------------------------------------
diff --git a/_docs/install/060-starting-the-web-ui.md b/_docs/install/060-starting-the-web-ui.md
index d37985f..5e8a885 100644
--- a/_docs/install/060-starting-the-web-ui.md
+++ b/_docs/install/060-starting-the-web-ui.md
@@ -3,13 +3,34 @@ title: Starting the Web Console
 parent: Install Drill
 ---
 
-The Drill Web Console is one of several [client interfaces](/docs/architecture-introduction/#drill-clients)
you can use to access Drill.  To access the Web Console in Drill 1.2 and later, go to `https://<IP
address>:8047`, where IP address is the host name or IP address of one of the installed
Drillbits in a distributed system. In Drill 1.1 and earlier, go to `http://<IP address>:8047`
to access the Web Console.
+The Drill Web Console is one of several [client interfaces](/docs/architecture-introduction/#drill-clients)
you can use to access Drill. 
+
+## Drill 1.1 and Earlier
+
+In Drill 1.1 and earlier, to open the Drill Web Console, launch a web browser, and go to
the following URL:
+
+`http://<IP address or host name>:8047` 
+
+where IP address is the host name or IP address of one of the installed Drillbits in a distributed
system or `localhost` in an embedded system.
+
+## Drill 1.2 and Later
+
+In Drill 1.2 and later, to open the Drill Web Console, launch a web browser, and go to one
of the following URLs depending on the configuration of HTTPS support:
+
+* `http://<IP address or host name>:8047`  
+  Use this URL when [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/#https-support)
is disabled (the default).
+* `https://<IP address or host name>:8047`  
+  Use this URL when HTTPS support is enabled.
+
+If HTTPS support is enabled, you need [authorization]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/)
to see and use the Storage tab of the Web Console. 
 
 If [user authentication]({{site.baseurl}}/docs/configuring-user-authentication/) is not enabled,
the Web Console controls appear: 
 
 ![Web Console]({{ site.baseurl }}/docs/img/web-ui.png)
 
-If [user authentication]({{site.baseurl}}/docs/configuring-user-authentication/) is enabled,
Drill 1.2 and later prompts you for a user name and password:
+Select the Storage tab to view, update, or add a new [storage plugin configuration]({{site.baseurl}}/docs/plugin-configuration-basics/).
+
+If [user authentication]({{site.baseurl}}/docs/configuring-user-authentication/) is enabled,
Drill prompts you for a user name and password:
 
 ![Web Console Login]({{ site.baseurl }}/docs/img/web-ui-login.png)
 

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/query-data/030-querying-hbase.md
----------------------------------------------------------------------
diff --git a/_docs/query-data/030-querying-hbase.md b/_docs/query-data/030-querying-hbase.md
index b6a96e4..d79f814 100644
--- a/_docs/query-data/030-querying-hbase.md
+++ b/_docs/query-data/030-querying-hbase.md
@@ -117,7 +117,7 @@ steps:
 [Start Drill]({{site.baseurl}}/docs/installing-drill-in-embedded-mode/) and complete the
following steps to query the HBase tables you created.
 
 1. Use the HBase storage plugin configuration.  
-    `USE HBase;`  
+    `USE hbase;`  
 2. Issue the following query to see the data in the students table:  
     `SELECT * FROM students;`  
     
@@ -206,7 +206,7 @@ The trivial example in the previous section queried little endian-encoded
data i
 
 ## Querying Big Endian-Encoded Data
 
-Drill optimizes scans of HBase tables when you use the ["CONVERT_TO and CONVERT_FROM data
types"]({{ site.baseurl }}/docs/supported-data-types/#convert_to-and-convert_from-data-types)
on big endian-encoded data. Drill provides the \*\_BE encoded types for use with CONVERT_TO
and CONVERT_FROM to take advantage of these optimizations. Here are a few examples of the
\*\_BE types.
+Drill optimizes scans of HBase tables when you use the ["CONVERT_TO and CONVERT_FROM data
types"]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
on big endian-encoded data. Drill provides the \*\_BE encoded types for use with CONVERT_TO
and CONVERT_FROM to take advantage of these optimizations. Here are a few examples of the
\*\_BE types.
 
 * DATE_EPOCH_BE  
 * TIME_EPOCH_BE  
@@ -233,10 +233,10 @@ To query HBase data:
 
 1. Connect the data source to Drill using the [HBase storage plugin]({{site.baseurl}}/docs/hbase-storage-plugin/).
 
 
-    `USE HBase;`
+    `USE hbase;`
 
 2. Determine the encoding of the HBase data you want to query. Ask the person in charge of
creating the data.  
-3. Based on the encoding type of the data, use the ["CONVERT_TO and CONVERT_FROM data types"]({{
site.baseurl }}/docs/supported-data-types/#convert_to-and-convert_from-data-types) to convert
HBase binary representations to an SQL type as you query the data.  
+3. Based on the encoding type of the data, use the ["CONVERT_TO and CONVERT_FROM data types"]({{
site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
to convert HBase binary representations to an SQL type as you query the data.  
     For example, use CONVERT_FROM in your Drill query to convert a big endian-encoded row
key to an SQL BIGINT type:  
 
     `SELECT CONVERT_FROM(BYTE_SUBSTR(row_key, 1, 8),'BIGINT_BE’) FROM my_hbase_table;`

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/query-data/050-querying-hive.md
----------------------------------------------------------------------
diff --git a/_docs/query-data/050-querying-hive.md b/_docs/query-data/050-querying-hive.md
index 8673525..64e9a23 100644
--- a/_docs/query-data/050-querying-hive.md
+++ b/_docs/query-data/050-querying-hive.md
@@ -10,7 +10,7 @@ To create a Hive table and query it with Drill, complete the following steps:
 
   1. Issue the following command to start the Hive shell:
   
-        hive
+      `hive`
   2. Issue the following command from the Hive shell create a table schema:
   
         hive> create table customers(FirstName string, LastName string, Company string,
Address string, City string, County string, State string, Zip string, Phone string, Fax string,
Email string, Web string) row format delimited fields terminated by ',' stored as textfile;
@@ -42,3 +42,8 @@ To create a Hive table and query it with Drill, complete the following steps:
         10 rows selected (1.5 seconds)
         0: jdbc:drill:schema=hiveremote>
 
+## Optimizing Reads of Parquet-Backed Tables
+
+Use the `store.hive.optimize_scan_with_native_readers` option to optimize reads of Parquet-backed
external tables from Hive. When set to TRUE, this option uses Drill native readers instead
of the Hive Serde interface, resulting in more performant queries of Parquet-backed external
tables. (Drill 1.2 and later)
+
+Set the `store.hive.optimize_scan_with_native_readers` option as described in the section,
["Planning and Execution Options"]({{site.baseurl}}/docs/planning-and-execution-options/).
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/sql-reference/data-types/010-supported-data-types.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/data-types/010-supported-data-types.md b/_docs/sql-reference/data-types/010-supported-data-types.md
index 3acc6bd..9279052 100644
--- a/_docs/sql-reference/data-types/010-supported-data-types.md
+++ b/_docs/sql-reference/data-types/010-supported-data-types.md
@@ -81,7 +81,7 @@ You do not assign a data type to every column name in a CREATE TABLE statement
t
 
 * [CAST]({{ site.baseurl }}/docs/data-type-conversion#cast)    
 * [CONVERT TO/FROM]({{ site.baseurl }}/docs/data-type-conversion#convert_to-and-convert_from)
  
-  Use the [CONVERT TO AND CONVERT FROM data types]({{ site.baseurl }}/docs/supported-data-types/#convert_to-and-convert_from-data-types)
 
+  Use the [CONVERT TO AND CONVERT FROM data types]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
 
 * Other [data conversion functions]({{ site.baseurl }}/docs/data-type-conversion#other-data-type-conversions)
  
 
 In some cases, Drill converts schema-less data to correctly-typed data implicitly. In this
case, you do not need to cast the data. The file format of the data and the nature of your
query determines the requirement for casting or converting. Differences in casting depend
on the data source. The following list describes how Drill treats data types from various
data sources:
@@ -132,7 +132,7 @@ In a textual file, such as CSV, Drill interprets every field as a VARCHAR,
as pr
 * [CAST]({{ site.baseurl }}/docs/data-type-conversion#cast)  
   Casts data from one data type to another.
 * CONVERT_TO and CONVERT_FROM functions
-  Converts data, including binary data, from one data type to another using ["CONVERT_TO
and CONVERT_FROM data types"]({{ site.baseurl }}/docs/supported-data-types/#convert_to-and-convert_from-data-types)
 
+  Converts data, including binary data, from one data type to another using ["CONVERT_TO
and CONVERT_FROM data types"]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
 
 * [TO_CHAR]({{ site.baseurl }}/docs/data-type-conversion/#to_char)  
   Converts a TIMESTAMP, INTERVALDAY/INTERVALYEAR, INTEGER, DOUBLE, or DECIMAL to a string.
 * [TO_DATE]({{ site.baseurl }}/docs/data-type-conversion/#to_date)  
@@ -190,7 +190,7 @@ If your FIXEDBINARY or VARBINARY data is in a format other than UTF-8,
or big-en
 
 \* Used to cast binary UTF-8 data coming to/from sources such as HBase. The CAST function
does not support all representations of FIXEDBINARY and VARBINARY. Only the UTF-8 format is
supported. 
 
-## CONVERT_TO and CONVERT_FROM Data Types
+## Data Types for CONVERT_TO and CONVERT_FROM Functions
 
 The [CONVERT_TO function]({{site.baseurl}}/docs/data-type-conversion/#convert_to-and-convert_from)
converts data to bytes from the input type. The [CONVERT_FROM function]({{site.baseurl}}/docs/data-type-conversion/#convert_to-and-convert_from)
converts data from bytes to the input type. For example, the following CONVERT_TO function
converts an integer to bytes using big endian encoding:
 
@@ -220,11 +220,14 @@ DATE_EPOCH| bytes(8)| DATE
 TIME_EPOCH_BE| bytes(8)| TIME  
 TIME_EPOCH| bytes(8)| TIME  
 TIMESTAMP_EPOCH| bytes(8)| DATE/TIME
+TIMESTAMP_IMPALA*| bytes(8)| INT96
 UTF8| bytes| VARCHAR  
 UTF16| bytes| VAR16CHAR  
 UINT8| bytes(8)| UINT8  
 UINT8_BE| bytes(8)| UINT8
 
+\* In Drill 1.2 and later, use the TIMESTAMP_IMPALA type with the CONVERT_FROM function to
decode a timestamp from Hive or Impala, as shown in the section, ["About INT96 Support"]({{site.baseurl}}/drill/docs/parquet-format/#about-int96-support).
+
 This table includes types such as INT, for converting little endian-encoded data and types
such as INT_BE for converting big endian-encoded data to Drill internal types. You need to
convert binary representations, such as data in HBase, to a Drill internal format as you query
the data. If you are unsure that the size of the source and destination INT or BIGINT you
are converting is the same, use CAST to convert these data types to/from binary.  
 
 \*\_HADOOPV in the data type name denotes the variable length integer as defined by Hadoop
libraries. Use a \*\_HADOOPV type if user data is encoded in this format by a Hadoop tool
outside MapR.

http://git-wip-us.apache.org/repos/asf/drill/blob/0cfc6b62/_docs/sql-reference/sql-functions/020-data-type-conversion.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-functions/020-data-type-conversion.md b/_docs/sql-reference/sql-functions/020-data-type-conversion.md
index ad7463a..2cac018 100644
--- a/_docs/sql-reference/sql-functions/020-data-type-conversion.md
+++ b/_docs/sql-reference/sql-functions/020-data-type-conversion.md
@@ -31,7 +31,7 @@ Use CONVERT_TO and CONVERT_FROM instead of the CAST function for converting
bina
 
 See the following tables for information about the data types to use for casting:
 
-* [CONVERT_TO and CONVERT_FROM Data Types]({{ site.baseurl }}/docs/supported-data-types/#convert_to-and-convert_from-data-types)
+* [CONVERT_TO and CONVERT_FROM Data Types]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
 * [Supported Data Types for Casting]({{ site.baseurl }}/docs/supported-data-types)
 * [Explicit Type Casting Maps]({{ site.baseurl }}/docs/supported-data-types/#explicit-type-casting-maps)
 
@@ -147,14 +147,14 @@ The CONVERT_TO and CONVERT_FROM functions convert binary data to/from
Drill inte
 
 *column* is the name of a column Drill reads.
 
-*type* is one of the encoding types listed in the [CONVERT_TO/FROM Data Types]({{ site.baseurl
}}/docs/data-types#convert_to-and-convert_from-data-types) table. 
+*type* is one of the encoding types listed in the [CONVERT_TO/FROM data types]({{ site.baseurl
}}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions) table.

 
 
 ### CONVERT_TO and CONVERT_FROM Usage Notes
 
 CONVERT_FROM and CONVERT_TO methods transform a known binary representation/encoding to a
Drill internal format. Use CONVERT_TO and CONVERT_FROM instead of the CAST function for converting
binary data types. CONVERT_TO/FROM functions work for data in a binary representation and
are more efficient to use than CAST. 
 
-Drill can optimize scans of HBase tables when you use the \*\_BE encoded types shown in section
 ["CONVERT_TO and CONVERT_FROM Data Types"]({{ site.baseurl }}/docs/supported-data-types/#convert_to-and-convert_from-data-types)
on big endian-encoded data. You need to use the HBase storage plugin and query data as described
in ["Querying Hbase"]({{ site.baseurl }}/docs/querying-hbase). To write Parquet binary data,
convert SQL data *to* binary data and store the data in a Parquet table while creating a table
as a selection (CTAS).
+Drill can optimize scans of HBase tables when you use the \*\_BE encoded types shown in section
 ["Data Types for CONVERT_TO and CONVERT_FROM Functions"]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
on big endian-encoded data. You need to use the HBase storage plugin and query data as described
in ["Querying Hbase"]({{ site.baseurl }}/docs/querying-hbase). To write Parquet binary data,
convert SQL data *to* binary data and store the data in a Parquet table while creating a table
as a selection (CTAS).
 
 CONVERT_TO also converts an SQL data type to complex types, including HBase byte arrays,
JSON and Parquet arrays, and maps. CONVERT_FROM converts from complex types, including HBase
arrays, JSON and Parquet arrays and maps to an SQL data type. 
 
@@ -178,7 +178,7 @@ This example shows how to use the CONVERT_FROM function to convert HBase
data to
     +-------------+---------------------+---------------------------------------------------------------------------+
     4 rows selected (1.335 seconds)
 
-You use the CONVERT_FROM function to decode the binary data, selecting a data type to use
from the [list of supported types]({{ site.baseurl }}/docs/data-type-conversion/#convert_to-and-convert_from-data-types).
JSON supports strings. To convert bytes to strings, use the UTF8 type:
+You use the CONVERT_FROM function to decode the binary data, selecting a data type to use
from the [list of supported types]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions).
JSON supports strings. To convert bytes to strings, use the UTF8 type:
 
     SELECT CONVERT_FROM(row_key, 'UTF8') AS studentid, 
            CONVERT_FROM(students.account.name, 'UTF8') AS name, 
@@ -265,7 +265,7 @@ This example assumes you are working in the Drill Sandbox. You modify
the `dfs`
 
 ### Convert the Binary HBase Students Table to JSON Data
 
-First, you set the storage format to JSON. Next, you use the CREATE TABLE AS (CTAS) statement
to convert from a selected file of a different format, HBase in this example, to the storage
format. You then convert the JSON file to Parquet using a similar procedure. Set the storage
format to Parquet, and use a CTAS statement to convert to Parquet from JSON. In each case,
you [select UTF8]({{ site.baseurl }}/docs/data-type-conversion/#convert_to-and-convert_from-data-types)
as the file format because the data you are converting from and then to consists of strings.
+First, you set the storage format to JSON. Next, you use the CREATE TABLE AS (CTAS) statement
to convert from a selected file of a different format, HBase in this example, to the storage
format. You then convert the JSON file to Parquet using a similar procedure. Set the storage
format to Parquet, and use a CTAS statement to convert to Parquet from JSON. In each case,
you [select UTF8]({{ site.baseurl }}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions)
as the file format because the data you are converting from and then to consists of strings.
 
 1. Start Drill on the Drill Sandbox and set the default storage format from Parquet to JSON.
 


Mime
View raw message