drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [1/2] drill git commit: consistent plugin language per Daniel
Date Fri, 17 Jul 2015 01:14:16 GMT
Repository: drill
Updated Branches:
  refs/heads/gh-pages 1b35859d1 -> 73fe89d8e


consistent plugin language per Daniel

remove redundancy by adding stuff here, wordsmith

fix broken link


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/2e5f3dd2
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/2e5f3dd2
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/2e5f3dd2

Branch: refs/heads/gh-pages
Commit: 2e5f3dd2c8998bcdf138aacffed3e7726da1b5e7
Parents: 1b35859
Author: Kristine Hahn <khahn@maprtech.com>
Authored: Wed Jul 8 16:02:11 2015 -0700
Committer: Kristine Hahn <khahn@maprtech.com>
Committed: Wed Jul 8 16:12:53 2015 -0700

----------------------------------------------------------------------
 .../010-connect-a-data-source-introduction.md   |  20 +---
 .../020-storage-plugin-registration.md          |  14 +--
 .../035-plugin-configuration-basics.md          |  61 ++++-------
 .../040-file-system-storage-plugin.md           | 102 ++++++++----------
 _docs/connect-a-data-source/050-workspaces.md   |  21 ++--
 .../060-hbase-storage-plugin.md                 |  19 +---
 .../070-hive-storage-plugin.md                  |  14 +--
 .../080-drill-default-input-format.md           |   6 +-
 .../090-mongodb-plugin-for-apache-drill.md      |   4 +-
 .../060-text-files-csv-tsv-psv.md               | 108 ++++++++++---------
 _docs/getting-started/010-drill-introduction.md |   2 +-
 11 files changed, 157 insertions(+), 214 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/010-connect-a-data-source-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/010-connect-a-data-source-introduction.md b/_docs/connect-a-data-source/010-connect-a-data-source-introduction.md
index b86542b..40bd15b 100644
--- a/_docs/connect-a-data-source/010-connect-a-data-source-introduction.md
+++ b/_docs/connect-a-data-source/010-connect-a-data-source-introduction.md
@@ -2,31 +2,19 @@
 title: "Connect a Data Source Introduction"
 parent: "Connect a Data Source"
 ---
-A storage plugin provides the following information to Drill:
+A storage plugin is a software module interface for connecting Drill to data sources. A storage
plugin typically optimizes execution of Drill queries, provides the location of the data,
and configures the workspace and file formats for reading data. Several storage plugins are
installed with Drill that you can configure to suit your environment. Through the storage
plugin, Drill connects to a data source, such as a database, a file on a local or distributed
file system, or a Hive metastore. 
 
-* Interfaces that Drill can use to read from and write to data sources.   
-* A set of storage plugin optimization rules that assist with efficient and faster execution
of Drill queries, such as pushdowns, statistics, and partition awareness.  
-
-Through the storage plugin, Drill connects to a data source, such as a database, a file on
a local or distributed file system, or a Hive metastore. When you execute a query, Drill gets
the plugin name in one of several ways:
+You can modify the default configuration of a storage plugin X and give the new version a
unique name Y. This document refers to Y as a different storage plugin, although it is actually
just a reconfiguration of original interface. When you execute a query, Drill gets the storage
plugin name in one of several ways:
 
 * The FROM clause of the query can identify the plugin to use.
 * The USE <plugin name> command can precede the query.
 * You can specify the storage plugin when starting Drill.
 
-In addition to providing a the connection string to the data source, the storage plugin configures
the workspace and file formats for reading data, as described in subsequent sections. 
-
-## Storage Plugins Internals
+## Storage Plugin Internals
 The following image represents the storage plugin layer between Drill and a
 data source:
 
 ![drill query flow]({{ site.baseurl }}/docs/img/storageplugin.png)
 
-A storage plugin provides the following information to Drill:
-
-  * Metadata available in the underlying data source
-  * Location of data
-  * Interfaces that Drill can use to read from and write to data sources
-  * A set of storage plugin optimization rules that assist with efficient and faster execution
of Drill queries, such as pushdowns, statistics, and partition awareness
-
-A storage plugin performs scanner and writer functions and informs the execution engine of
any native capabilities, such
+In addition to the previously mentioned functions, a storage plugin performs scanner and
writer functions and informs the execution engine of any native capabilities, such
 as predicate pushdown, joins, and SQL.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/020-storage-plugin-registration.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/020-storage-plugin-registration.md b/_docs/connect-a-data-source/020-storage-plugin-registration.md
index 1a0113a..73ff050 100644
--- a/_docs/connect-a-data-source/020-storage-plugin-registration.md
+++ b/_docs/connect-a-data-source/020-storage-plugin-registration.md
@@ -2,25 +2,25 @@
 title: "Storage Plugin Registration"
 parent: "Connect a Data Source"
 ---
-You connect Drill to a file system, Hive, HBase, or other data source using storage plugins.
Drill includes a number of storage plugins in the installation. On the Storage tab of the
Web UI, you can view, create, reconfigure, and register a storage plugin. To open the Storage
tab, go to `http://<IP address>:8047/storage`, where IP address is any one of the installed
drill bits:
+You connect Drill to a file system, Hive, HBase, or other data source through a storage plugin.
On the Storage tab of the Web UI, you can view and reconfigure a storage plugin. You can create
a new name for the reconfigured version, thereby registering the new version. To open the
Storage tab, go to `http://<IP address>:8047/storage`, where IP address is any one of
the installed Drillbits in a distributed system or `localhost` in an embedded system:
 
 ![drill-installed plugins]({{ site.baseurl }}/docs/img/plugin-default.png)
 
-The Drill installation registers the `cp`, `dfs`, `hbase`, `hive`, and `mongo` storage plugins
instances by default.
+The Drill installation registers the the `cp`, `dfs`, `hbase`, `hive`, and `mongo` storage
plugin configurations.
 
 * `cp`  
   Points to a JAR file in the Drill classpath that contains the Transaction Processing Performance
Council (TPC) benchmark schema TPC-H that you can query. 
 * `dfs`  
-  Points to the local file system on your machine, but you can configure this instance to
+  Points to the local file system, but you can configure this storage plugin to
 point to any distributed file system, such as a Hadoop or S3 file system. 
 * `hbase`  
-   Provides a connection to HBase/M7.
+   Provides a connection to HBase.
 * `hive`  
-   Integrates Drill with the Hive metadata abstraction of files, HBase/M7, and libraries
to read data and operate on SerDes and UDFs.
+   Integrates Drill with the Hive metadata abstraction of files, HBase, and libraries to
read data and operate on SerDes and UDFs.
 * `mongo`  
    Provides a connection to MongoDB data.
 
-In the Drill sandbox,  the `dfs` storage plugin connects you to the MapR File System (MFS).
Using an installation of Drill instead of the sandbox, `dfs` connects you to the root of your
file system.
+In the [Drill sandbox]({{site.baseurl}}/docs/about-the-mapr-sandbox/), the `dfs` storage
plugin connects you to a simulation of a distributed file system. If you install Drill, `dfs`
connects you to the root of your file system.
 
-Storage plugin configurations are saved in a temporary directory (embedded mode) or in ZooKeeper
(distributed mode). Seeing a storage plugin that you created in one version appear in the
Drill Web UI of another version is expected. For example, on Mac OS X, Drill uses `/tmp/drill/sys.storage_plugins`
to store storage plugin configurations. To revert to the default storage plugins for a particular
version, in embedded mode, delete the contents of this directory and restart the Drill shell.
+Drill saves storage plugin configurations in a temporary directory (embedded mode) or in
ZooKeeper (distributed mode). The storage plugin configuration persists after upgrading, so
a configuration that you created in one version of Drill appears in the Drill Web UI of an
upgraded version of Drill. For example, on Mac OS X, Drill uses `/tmp/drill/sys.storage_plugins`
to store storage plugin configurations. To revert to the default storage plugins for a particular
version, in embedded mode, delete the contents of this directory and restart the Drill shell.
 

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/035-plugin-configuration-basics.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/035-plugin-configuration-basics.md b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
index b5b6c68..8b89241 100644
--- a/_docs/connect-a-data-source/035-plugin-configuration-basics.md
+++ b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
@@ -2,29 +2,27 @@
 title: "Plugin Configuration Basics"
 parent: "Storage Plugin Configuration"
 ---
-When you add or update storage plugin instances on one Drill node in a Drill
-cluster, Drill broadcasts the information to other Drill nodes 
+When you add or update storage plugin instances on one Drill node in a 
+cluster having multiple installations of Drill, Drill broadcasts the information to other
Drill nodes 
 to synchronize the storage plugin configurations. You do not need to
 restart any of the Drillbits when you add or update a storage plugin instance.
 
-Use the Drill Web UI to update or add a new storage plugin. Launch a web browser, go to:
`http://<IP address or host name>:8047`, and then go to the Storage tab. 
+Use the Drill Web UI to update or add a new storage plugin configuration. Launch a web browser,
go to: `http://<IP address or host name>:8047`, and then go to the Storage tab. 
 
-To create and configure a new storage plugin:
+To create a name and new configuration:
 
-1. Enter a storage name in New Storage Plugin.
-   Each storage plugin registered with Drill must have a distinct
+1. Enter a name in **New Storage Plugin**.
+   Each configuration registered with Drill must have a distinct
 name. Names are case-sensitive.
-2. Click Create.  
-3. In Configuration, configure attributes of the storage plugin, if applicable, using JSON
formatting. The Storage Plugin Attributes table in the next section describes attributes typically
reconfigured by users. 
-4. Click Create.
-
-Click Update to reconfigure an existing, enabled storage plugin.
+2. Click **Create**.  
+3. In Configuration, it is recommended that you modify a copy of an existing configuration
if possible. Reconfigure attributes of the storage plugin using JSON formatting. The Storage
Plugin Attributes table in the next section describes attributes typically reconfigured by
users. 
+4. Click **Create**.
 
 ## Storage Plugin Attributes
-The following graphic shows key attributes of a typical dfs storage plugin:  
+The following graphic shows key attributes of a typical `dfs`-based storage plugin configuration:
 
 ![dfs plugin]({{ site.baseurl }}/docs/img/connect-plugin.png)
 ## List of Attributes and Definitions
-The following table describes the attributes you configure for storage plugins. 
+The following table describes the attributes you configure for storage plugins installed
with Drill. 
 <table>
   <tr>
     <th>Attribute</th>
@@ -135,7 +133,7 @@ The following table describes the attributes you configure for storage
plugins.
 
 ## Using the Formats
 
-You can use the following attributes when the `sys.options` property setting `exec.storage.enable_new_text_reader`
is true (the default):
+You can use the following attributes in the `formats` area of the storage plugin configuration.
When setting these attributes, you also need to set the `sys.options` property `exec.storage.enable_new_text_reader`
to true (the default):
 
 * comment  
 * escape  
@@ -143,30 +141,11 @@ You can use the following attributes when the `sys.options` property
setting `ex
 * quote  
 * skipFirstLine
 
-The "formats" apply to all workspaces defined in a storage plugin. A typical use case defines
separate storage plugins for different root directories to query the files stored below the
directory. An alternative use case defines multiple formats within the same storage plugin
and names target files using different extensions to match the formats.
-
-The following example of a storage plugin for reading CSV files with the new text reader
includes two formats for reading files having either a `csv` or `csv2` extension. The text
reader does include the first line of column names in the queries of `.csv` files but does
not include it in queries of `.csv2` files. 
-
-    "csv": {
-      "type": "text",
-      "extensions": [
-        "csv"
-      ],  
-      "delimiter": "," 
-    },  
-    "csv_with_header": {
-      "type": "text",
-      "extensions": [
-        "csv2"
-      ],  
-      "comment": "&",
-      "skipFirstLine": true,
-      "delimiter": "," 
-    },  
+For more information and examples of using formats for text files, see ["Text Files: CSV,
TSV, PSV"]({{site.baseurl}}{{site.baseurl}}/docs/text-files-csv-tsv-psv/)
 
 ## Using Other Attributes
 
-The configuration of other attributes, such as `size.calculator.enabled` in the hbase plugin
and `configProps` in the hive plugin, are implementation-dependent and beyond the scope of
this document.
+The configuration of other attributes, such as `size.calculator.enabled` in the `hbase` plugin
and `configProps` in the `hive` plugin, are implementation-dependent and beyond the scope
of this document.
 
 ## Case-sensitive Names
 As previously mentioned, workspace and storage plugin names are case-sensitive. For example,
the following query uses a storage plugin name `dfs` and a workspace name `clicks`. When you
refer to `dfs.clicks` in an SQL statement, use the defined case:
@@ -177,15 +156,15 @@ For example, using uppercase letters in the query after defining the
storage plu
 
 ## Storage Plugin REST API
 
-Drill provides a REST API that you can use to create a storage plugin. Use an HTTP POST and
pass two properties:
+Drill provides a REST API that you can use to create a storage plugin configuration. Use
an HTTP POST and pass two properties:
 
 * name  
-  The plugin name. 
+  The storage plugin configuration name. 
 
 * config  
-  The storage plugin definition as you would enter it in the Web UI.
+  The attribute settings as you would enter it in the Web UI.
 
-For example, this command creates a plugin named myplugin for reading files of an unknown
type located on the root of the file system:
+For example, this command creates a storage plugin named myplugin for reading files of an
unknown type located on the root of the file system:
 
     curl -X POST -/json" -d '{"name":"myplugin", "config": {"type": "file", "enabled": false,
"connection": "file:///", "workspaces": { "root": { "location": "/", "writable": false, "defaultInputFormat":
null}}, "formats": null}}' http://localhost:8047/storage/myplugin.json
 
@@ -196,8 +175,8 @@ If you need to add a storage plugin to Drill and do not want to use a
web browse
 Bootstrapping a storage plugin works only when the first drillbit in the cluster first starts
up. The configuration is
 stored in zookeeper, preventing Drill from picking up the boostrap-storage-plugins.json again.
 
-After cluster startup, you have to use the REST API or Drill Web UI to add a storage plugin.
Alternatively, you
+After cluster startup, you have to use the REST API or Drill Web UI to add a storage plugin
configuration. Alternatively, you
 can modify the entry in zookeeper by uploading the json file for
-that plugin to the /drill directory of the zookeeper installation, or just delete the /drill
directory if you do not have configuration properties to preserve.
+that plugin to the /drill directory of the zookeeper installation, or by just deleting the
/drill directory if you do not have configuration properties to preserve.
 
 If you configure an HBase storage plugin using bootstrap-storage-plugins.json file and HBase
is not installed, you might experience a delay when executing the queries. Configure the [HBase
client timeout](http://hbase.apache.org/book.html#config.files) and retry settings in the
config block of HBase plugin instance configuration.

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/040-file-system-storage-plugin.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/040-file-system-storage-plugin.md b/_docs/connect-a-data-source/040-file-system-storage-plugin.md
index 4753784..7d299d2 100644
--- a/_docs/connect-a-data-source/040-file-system-storage-plugin.md
+++ b/_docs/connect-a-data-source/040-file-system-storage-plugin.md
@@ -9,69 +9,57 @@ system on your machine by default.
 
 ## Connecting Drill to a File System
 
-In a Drill cluster, you typically do not query the local file system, but instead place files
on the distributed file system. You configure the connection property of the storage plugin
workspace to connect Drill to a distributed file system. For example, the following connection
properties connect Drill to an HDFS or MapR-FS cluster from a client:
+In a Drill cluster, you typically do not query the local file system, but instead place files
on the distributed file system. You configure the connection property of the storage plugin
workspace to connect Drill to a distributed file system. For example, the following connection
properties connect Drill to an HDFS cluster from a client:
 
-* HDFS  
-  `"connection": "hdfs://<IP Address>:<Port>/"`  
-* MapR-FS Remote Cluster  
-  `"connection": "maprfs://<IP Address>/"`  
+`"connection": "hdfs://<IP Address>:<Port>/"`   
 
-To query a file on HDFS from a node on the cluster, you can simply change the connection
to from `file:///` to `hdfs:///` in the `dfs` storage plugin.
+To query a file on HDFS from a node on the cluster, you can simply change the connection
to from `file:///` to `hdfs://` in the `dfs` storage plugin.
 
-To register a local or a distributed file system with Apache Drill, complete
-the following steps:
+To change the `dfs` storage plugin configuration to point to a local or a distributed file
system, use `connection` attributes as shown in the following example.
+* Local file system example:
 
-  1. Navigate to http://localhost:8047, and select the **Storage** tab.
-  2. In the New Storage Plugin window, enter a unique name and then click **Create**.
-  3. In the Configuration window, provide the following configuration information for the
type of file system that you are configuring as a data source.
-     * Local file system example:
-
-            {
-              "type": "file",
-              "enabled": true,
-              "connection": "file:///",
-              "workspaces": {
-                "root": {
-                  "location": "/user/max/donuts",
-                  "writable": false,
-                  "defaultInputFormat": null
-                 }
-              },
-                 "formats" : {
-                   "json" : {
-                     "type" : "json"
-                   }
-                 }
-              }
-     * Distributed file system example:
+    {
+      "type": "file",
+      "enabled": true,
+      "connection": "file:///",
+      "workspaces": {
+        "root": {
+          "location": "/user/max/donuts",
+          "writable": false,
+          "defaultInputFormat": null
+         }
+      },
+         "formats" : {
+           "json" : {
+             "type" : "json"
+           }
+         }
+      }
+* Distributed file system example:
     
-            {
-              "type" : "file",
-              "enabled" : true,
-              "connection" : "hdfs://10.10.30.156:8020/",
-              "workspaces" : {
-                "root" : {
-                  "location" : "/user/root/drill",
-                  "writable" : true,
-                  "defaultInputFormat" : null
-                }
-              },
-              "formats" : {
-                "json" : {
-                  "type" : "json"
-                }
-              }
-            }
-
-      To connect to a Hadoop file system, you include the IP address of the
+    {
+      "type" : "file",
+      "enabled" : true,
+      "connection" : "hdfs://10.10.30.156:8020/",
+      "workspaces" : {
+        "root" : {
+          "location" : "/user/root/drill",
+          "writable" : true,
+          "defaultInputFormat" : null
+        }
+      },
+      "formats" : {
+        "json" : {
+          "type" : "json"
+        }
+      }
+    }
+
+To connect to a Hadoop file system, you include the IP address of the
 name node and the port number.
-  4. Click **Enable**.
-
-After you have configured a storage plugin instance for the file system, you
-can issue Drill queries against it.
 
-The following example shows an instance of a file type storage plugin with a
-workspace named `json_files` configured to point Drill to the
+The following example shows an file type storage plugin configuration with a
+workspace named `json_files`. The configuration points Drill to the
 `/users/max/drill/json/` directory in the local file system `(dfs)`:
 
     {
@@ -86,7 +74,7 @@ workspace named `json_files` configured to point Drill to the
        } 
     },
 
-{% include startnote.html %}The `connection` parameter in the configuration above is "`file:///`",
connecting Drill to the local file system (`dfs`).{% include endnote.html %}
+The `connection` parameter in this configuration is "`file:///`", connecting Drill to the
local file system (`dfs`).
 
 To query a file in the example `json_files` workspace, you can issue the `USE`
 command to tell Drill to use the `json_files` workspace configured in the `dfs`

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/050-workspaces.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/050-workspaces.md b/_docs/connect-a-data-source/050-workspaces.md
index b1156c9..b535267 100644
--- a/_docs/connect-a-data-source/050-workspaces.md
+++ b/_docs/connect-a-data-source/050-workspaces.md
@@ -2,19 +2,17 @@
 title: "Workspaces"
 parent: "Storage Plugin Configuration"
 ---
-When you register an instance of a file system data source, you can configure
-one or more workspaces for the instance. The workspace defines the  directory location of
files in a local or distributed file system. Drill searches the workspace to locate data when
+You can define one or more workspaces in a storage plugin configuration. The workspace defines
the directory location of files in a local or distributed file system. Drill searches the
workspace to locate data when
 you run a query. The `default`
 workspace points to the root of the file system. 
 
-Configuring `workspaces` in the storage plugin definition to include the file location simplifies
the query, which is important when querying the same data source repeatedly. After you configure
a long path name in the workspaces location property, instead of
+Configuring `workspaces` to include a file location simplifies the query, which is important
when querying the same data source repeatedly. After you configure a long path name in the
workspaces location property, instead of
 using the full path to the data source, you use dot notation in the FROM
 clause.
 
 ``<workspaces>.`<location>```
 
-To query the data source while you are _not_ connected to
-that storage plugin, include the plugin name. This syntax assumes you did not issue a USE
statement to connect to a storage plugin that defines the
+To query the data source while you are not *using* that storage plugin, include the plugin
name. This syntax assumes you did not issue a USE statement to connect to a storage plugin
that defines the
 location of the data:
 
 ``<plugin>.<workspaces>.`<location>```
@@ -22,14 +20,14 @@ location of the data:
 
 ## No Workspaces for Hive and HBase
 
-You cannot create workspaces for
-`hive` and `hbase` storage plugins, though Hive databases show up as workspaces in
+You cannot configure workspaces for
+`hive` and `hbase`, though Hive databases show up as workspaces in
 Drill. Each `hive` instance includes a `default` workspace that points to the  Hive metastore.
When you query
 files and tables in the `hive default` workspaces, you can omit the
 workspace name from the query.
 
 For example, you can issue a query on a Hive table in the `default workspace`
-using either of the following formats and get the same results:
+using either of the following queries and get the same results:
 
 **Example**
 
@@ -38,11 +36,8 @@ using either of the following formats and get the same results:
 
 {% include startnote.html %}Default is a reserved word. You must enclose reserved words in
back ticks.{% include endnote.html %}
 
-Because HBase instances do not have workspaces, you can use the following
-format to query a table in HBase:
+Because the HBase storage plugin configuration does not have a workspace, you can use the
following
+query:
 
     SELECT * FROM hbase.customers LIMIT 10;
 
-After you register a data source as a storage plugin instance with Drill, and
-optionally configure workspaces, you can query the data source.
-

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/060-hbase-storage-plugin.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/060-hbase-storage-plugin.md b/_docs/connect-a-data-source/060-hbase-storage-plugin.md
index e6a1ef2..488a564 100644
--- a/_docs/connect-a-data-source/060-hbase-storage-plugin.md
+++ b/_docs/connect-a-data-source/060-hbase-storage-plugin.md
@@ -2,19 +2,10 @@
 title: "HBase Storage Plugin"
 parent: "Storage Plugin Configuration"
 ---
-Register a storage plugin instance and specify a ZooKeeper quorum to connect
-Drill to an HBase data source. When you register a storage plugin instance for
-an HBase data source, provide a unique name for the instance, and identify the
-type as “hbase” in the Drill Web UI.
+Specify a ZooKeeper quorum to connect
+Drill to an HBase data source. Drill supports HBase version 0.98.
 
-Drill supports HBase version 0.98.
-
-To register HBase with Drill, complete the following steps:
-
-  1. Navigate to [http://localhost:8047](http://localhost:8047/), and select the **Storage**
tab
-  2. In the disabled storage plugins section, click **Update** next to the `hbase` instance.
-  3. In the Configuration window, specify the ZooKeeper quorum and port. 
-  
+To HBase storage plugin configuration installed with Drill appears as follows when you navigate
to [http://localhost:8047](http://localhost:8047/), and select the **Storage** tab.
 
      **Example**  
 
@@ -28,7 +19,3 @@ To register HBase with Drill, complete the following steps:
               "enabled": true
             }
 
-  4. Click **Enable**.
-
-After you configure a storage plugin instance for the HBase, you can
-issue Drill queries against it.

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/070-hive-storage-plugin.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/070-hive-storage-plugin.md b/_docs/connect-a-data-source/070-hive-storage-plugin.md
index 753aa0d..c7ab31f 100644
--- a/_docs/connect-a-data-source/070-hive-storage-plugin.md
+++ b/_docs/connect-a-data-source/070-hive-storage-plugin.md
@@ -2,12 +2,6 @@
 title: "Hive Storage Plugin"
 parent: "Storage Plugin Configuration"
 ---
-You can register a storage plugin instance that connects Drill to a Hive data
-source that has a remote or embedded metastore service. When you register a
-storage plugin instance for a Hive data source, provide a unique name for the
-instance, and identify the type as “`hive`”. You must also provide the
-metastore connection information.
-
 Drill 1.0 supports Hive 0.13. Drill 1.1 supports Hive 1.0. To access Hive tables
 using custom SerDes or InputFormat/OutputFormat, all nodes running Drillbits
 must have the SerDes or InputFormat/OutputFormat `JAR` files in the 
@@ -23,7 +17,7 @@ in the Drill Web UI to configure a connection to Drill.
 
 {% include startnote.html %}Verify that the Hive metastore service is running before you
register the Hive metastore.{% include endnote.html %}  
 
-To register a remote Hive metastore with Drill, complete the following steps:
+To configure a remote Hive metastore, complete the following steps:
 
 1. Issue the following command to start the Hive metastore service on the system specified
in the `hive.metastore.uris`:
    `hive --service metastore`
@@ -64,10 +58,10 @@ After configuring a Hive storage plugin, you can [query Hive tables]({{
site.bas
 
 In this configuration, the Hive metastore is embedded within the Drill process. Configure
an embedded metastore only in a cluster that runs a single Drillbit and only for testing purposes.
Do not embed the Hive metastore in production systems.
 
-Provide the metastore database configuration settings in the Drill Web UI. Before you register
Hive, verify that the driver you use to connect to the Hive metastore is in the Drill classpath
located in `/<drill installation directory>/lib/.` If the driver is not there, copy
the driver to `/<drill
+Provide the metastore database configuration settings in the Drill Web UI. Before you configure
an embedded Hive metastore, verify that the driver you use to connect to the Hive metastore
is in the Drill classpath located in `/<drill installation directory>/lib/.` If the
driver is not there, copy the driver to `/<drill
 installation directory>/lib` on the Drill node. For more information about storage types
and configurations, refer to ["Hive Metastore Administration"](https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin).
 
-To register an embedded Hive metastore with Drill, complete the following
+To configure an embedded Hive metastore, complete the following
 steps:
 
 1. Navigate to `http://<host>:8047`, and select the **Storage** tab.
@@ -87,6 +81,6 @@ steps:
               "hive.metastore.sasl.enabled": "false"
             }
           }
-5. Change the `"fs.default.name":` attribute to specify the default location of files. The
value needs to be a URI that is available and capable of handling filesystem requests. For
example, change the local file system URI `"file:///"` to the HDFS URI: `hdfs://`, or to the
path on HDFS with a namenode: `hdfs://<authority>:<port>`
+5. Change the `"fs.default.name":` attribute to specify the default location of files. The
value needs to be a URI that is available and capable of handling file system requests. For
example, change the local file system URI `"file:///"` to the HDFS URI: `hdfs://`, or to the
path on HDFS with a namenode: `hdfs://<authority>:<port>`
 6. Click **Enable**.
   
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/080-drill-default-input-format.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/080-drill-default-input-format.md b/_docs/connect-a-data-source/080-drill-default-input-format.md
index f09f1df..960512d 100644
--- a/_docs/connect-a-data-source/080-drill-default-input-format.md
+++ b/_docs/connect-a-data-source/080-drill-default-input-format.md
@@ -38,8 +38,8 @@ steps:
 
   1. Navigate to the Drill Web UI at `<drill_node_ip_address>:8047`. The Drillbit process
must be running on the node before you connect to the Drill Web UI.
   2. Select **Storage** in the toolbar.
-  3. Click **Update** next to the file system for which you want to define a default input
format for a workspace.
-  4. In the Configuration area, locate the workspace for which you would like to define the
default input format, and change the `defaultInputFormat` attribute to any of the supported
file types.
+  3. Click **Update** next to the storage plugin for which you want to define a default input
format for a workspace.
+  4. In the Configuration area, locate the workspace, and change the `defaultInputFormat`
attribute to any of the supported file types.
 
      **Example**
      
@@ -61,4 +61,4 @@ steps:
 
 ## Querying Compressed Files
 
-You can query compressed GZ files, such as JSON and CSV, as well as uncompressed files. The
file extension specified in the `formats . . . extensions` property of the storage plugin
must precede the gz extension in the file name. For example, `proddata.json.gz` or `mydata.csv.gz`
are valid file names to use in a query, as shown in the example in ["Querying the GZ File
Directly"]({{site.baseurl"}}/docs/querying-plain-text-files/#query-the-gz-file-directly).
+You can query compressed GZ files, such as JSON and CSV, as well as uncompressed files. The
file extension specified in the `formats . . . extensions` property of the storage plugin
configuration must precede the gz extension in the file name. For example, `proddata.json.gz`
or `mydata.csv.gz` are valid file names to use in a query, as shown in the example in ["Querying
the GZ File Directly"]({{site.baseurl"}}/docs/querying-plain-text-files/#query-the-gz-file-directly).

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/connect-a-data-source/090-mongodb-plugin-for-apache-drill.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/090-mongodb-plugin-for-apache-drill.md b/_docs/connect-a-data-source/090-mongodb-plugin-for-apache-drill.md
index 6a7b56d..afd6ee2 100644
--- a/_docs/connect-a-data-source/090-mongodb-plugin-for-apache-drill.md
+++ b/_docs/connect-a-data-source/090-mongodb-plugin-for-apache-drill.md
@@ -23,7 +23,7 @@ provided by MongoDB that you download in the following steps:
 
 ## Configuring MongoDB
 
-Start Drill and configure the MongoDB storage plugin instance in the Drill Web
+Start Drill and configure the MongoDB storage plugin in the Drill Web
 UI to connect to Drill. Drill must be running in order to access the Web UI.
 
 Complete the following steps to configure MongoDB as a data source for Drill:
@@ -41,7 +41,7 @@ Complete the following steps to configure MongoDB as a data source for Drill:
         {
           "type": "mongo",
           "connection": "mongodb://localhost:27017/",
-          "enabled": true
+          "enabled": false
         }
 
      {% include startnote.html %}27017 is the default port for `mongodb` instances.{% include
endnote.html %} 

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md
----------------------------------------------------------------------
diff --git a/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md b/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md
index 5b0d874..11ddb69 100644
--- a/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md
+++ b/_docs/data-sources-and-file-formats/060-text-files-csv-tsv-psv.md
@@ -44,56 +44,14 @@ In the storage plugin configuration, you can set the following attributes
that a
 * boolean skipFirstLine = false;  
   Set to true to avoid reading headers as data. 
 
-For more information about storage plugin configuration, see ["List of Attributes and Definitions"]({{site.baseurl}}/docs/plugin-configuration-basics/#list-of-attributes-and-definitions).
+Set the `sys.options` property setting `exec.storage.enable_new_text_reader` to true (the
default) before attempting to use these attributes:
 
-You can deal with a mix of text files with and without headers either by creating two separate
format plugins or by creating two format plugins within the same storage plugin. The former
approach is typically easier than the latter.
-
-### Creating Two Separate Format Plugins
-Format plugins are associated with a particular storage plugin. Storage plugins define a
root directory that Drill targets when using the storage plugin. You can define separate storage
plugins for different root directories, and define each of the format attributes to match
the files stored below that directory. All files can use the .csv extension, as shown in the
following example:
-
-Storage Plugin A
-
-    "csv": {
-      "type": "text",
-      "extensions": [
-        "csv"
-      ],
-      "delimiter": ","
-    },
-    . . .
-
-
-Storage Plugin B
-
-    "csv": {
-      "type": "text",
-      "extensions": [
-        "csv"
-      ],
-      "comment": "&",
-      "skipFirstLine": true,
-      "delimiter": ","
-    },
+* comment  
+* escape  
+* fieldDeliimiter  
+* quote  
+* skipFirstLine
 
-### Creating Two Format Plugins within the Same Storage Plugin
-Give a different extension to files with a header and to files without a header, and use
a storage plugin that looks something like the following example. This method requires renaming
some files to use the csv2 extension, as shown in the following example:
-
-    "csv": {
-      "type": "text",
-      "extensions": [
-        "csv"
-      ],
-      "delimiter": ","
-    },
-    "csv_with_header": {
-      "type": "text",
-      "extensions": [
-        "csv2"
-      ],
-      "comment": "&",
-      "skipFirstLine": true,
-      "delimiter": ","
-    },
 
 ## Examples of Querying Text Files
 The examples in this section show the results of querying CSV files that use and do not use
a header, include comments, and use an escape character:
@@ -168,3 +126,57 @@ The examples in this section show the results of querying CSV files that
use and
     | ["hello","1","2","3"]  |
     +------------------------+
     7 rows selected (0.111 seconds)
+
+## Strategies for Using Attributes
+The attributes, such as skipFirstLine, apply to all workspaces defined in a storage plugin.
A typical use case defines separate storage plugins for different root directories to query
the files stored below the directory. An alternative use case defines multiple formats within
the same storage plugin and names target files using different extensions to match the formats.
+
+You can deal with a mix of text files with and without headers either by creating two separate
format plugins or by creating two format plugins within the same storage plugin. The former
approach is typically easier than the latter.
+
+### Creating Two Separate Storage Plugin Configurations
+A storage plugin configuration defines a root directory that Drill targets. You can use a
different configuration for each root directory that sets attributes to match the files stored
below that directory. All files can use the same extension, such as .csv, as shown in the
following example:
+
+Storage Plugin A
+
+    "csv": {
+      "type": "text",
+      "extensions": [
+        "csv"
+      ],
+      "delimiter": ","
+    },
+    . . .
+
+
+Storage Plugin B
+
+    "csv": {
+      "type": "text",
+      "extensions": [
+        "csv"
+      ],
+      "comment": "&",
+      "skipFirstLine": true,
+      "delimiter": ","
+    },
+
+### Creating One Storage Plugin Configuration to Handle Multiple Formats
+You can use a different extension for files with and without a header, and use a storage
plugin that looks something like the following example. This method requires renaming some
files to use the csv2 extension.
+
+    "csv": {
+      "type": "text",
+      "extensions": [
+        "csv"
+      ],
+      "delimiter": ","
+    },
+    "csv_with_header": {
+      "type": "text",
+      "extensions": [
+        "csv2"
+      ],
+      "comment": "&",
+      "skipFirstLine": true,
+      "delimiter": ","
+    },
+
+

http://git-wip-us.apache.org/repos/asf/drill/blob/2e5f3dd2/_docs/getting-started/010-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/getting-started/010-drill-introduction.md b/_docs/getting-started/010-drill-introduction.md
index 1ccfb31..31c9ed0 100644
--- a/_docs/getting-started/010-drill-introduction.md
+++ b/_docs/getting-started/010-drill-introduction.md
@@ -15,7 +15,7 @@ Many enhancements in Apache Drill 1.1 include the following key features:
 
 * [SQL window functions]({{site.baseurl}}/docs/sql-window-functions)
 * [Automatic partitioning]({{site.baseurl}}) using the new [PARTITION BY]({{site.baseurl}}/docs/partition-by-clause)
clause in the CTAS command
-* [Delegated Hive impersonation](({{site.baseurl}}/docs/configuring-user-impersonation-with-hive-authorization/)
+* [Delegated Hive impersonation]({{site.baseurl}}/docs/configuring-user-impersonation-with-hive-authorization/)
 * Support for UNION and UNION ALL and better optimized plans that include UNION.
 
 ## What's New in Apache Drill 1.0


Mime
View raw message