In the storage plugin configuration, you can set the following attributes that affect how Drill reads CSV, TSV, PSV (comma-, tab-, pipe-separated) files.

In the storage plugin configuration, you set the attributes that affect how Drill reads CSV, TSV, PSV (comma-, tab-, pipe-separated) files:

Set the sys.options property setting exec.storage.enable_new_text_reader to true (the default) before attempting to use these attributes.

You can deal with a mix of text files with and without headers either by creating two separate format plugins or by creating two format plugins within the same storage plugin. The former approach is typically easier than the latter.

Creating Two Separate Format Plugins

Format plugins are associated with a particular storage plugin. Storage plugins define a root directory that Drill targets when using the storage plugin. You can define separate storage plugins for different root directories, and define each of the format attributes to match the files stored below that directory. All files can use the .csv extension, as shown in the following example:

Creating Two Format Plugins within the Same Storage Plugin

Give a different extension to files with a header and to files without a header, and use a storage plugin that looks something like the following example. This method requires renaming some files to use the csv2 extension, as shown in the following example:

Examples of Querying Text Files

The examples in this section show the results of querying CSV files that use and do not use a header, include comments, and use an escape character:

Strategies for Using Attributes

The attributes, such as skipFirstLine, apply to all workspaces defined in a storage plugin. A typical use case defines separate storage plugins for different root directories to query the files stored below the directory. An alternative use case defines multiple formats within the same storage plugin and names target files using different extensions to match the formats.

Creating Two Separate Storage Plugin Configurations

A storage plugin configuration defines a root directory that Drill targets. You can use a different configuration for each root directory that sets attributes to match the files stored below that directory. All files can use the same extension, such as .csv, as shown in the following example:

Creating One Storage Plugin Configuration to Handle Multiple Formats

You can use a different extension for files with and without a header, and use a storage plugin that looks something like the following example. This method requires renaming some files to use the csv2 extension.

http://git-wip-us.apache.org/repos/asf/drill-site/blob/d0ba86f3/docs/workspaces/index.html ---------------------------------------------------------------------- diff --git a/docs/workspaces/index.html b/docs/workspaces/index.html index 96e001f..b1bfe78 100644 --- a/docs/workspaces/index.html +++ b/docs/workspaces/index.html @@ -1003,33 +1003,31 @@

When you register an instance of a file system data source, you can configure -one or more workspaces for the instance. The workspace defines the directory location of files in a local or distributed file system. Drill searches the workspace to locate data when +

You can define one or more workspaces in a storage plugin configuration. The workspace defines the directory location of files in a local or distributed file system. Drill searches the workspace to locate data when you run a query. The default workspace points to the root of the file system.

Configuring workspaces in the storage plugin definition to include the file location simplifies the query, which is important when querying the same data source repeatedly. After you configure a long path name in the workspaces location property, instead of +

Configuring workspaces to include a file location simplifies the query, which is important when querying the same data source repeatedly. After you configure a long path name in the workspaces location property, instead of using the full path to the data source, you use dot notation in the FROM clause.

<workspaces>.`<location>`

To query the data source while you are not connected to -that storage plugin, include the plugin name. This syntax assumes you did not issue a USE statement to connect to a storage plugin that defines the +

To query the data source while you are not using that storage plugin, include the plugin name. This syntax assumes you did not issue a USE statement to connect to a storage plugin that defines the location of the data:

<plugin>.<workspaces>.`<location>`

No Workspaces for Hive and HBase

You cannot create workspaces for -hive and hbase storage plugins, though Hive databases show up as workspaces in +

You cannot configure workspaces for +hive and hbase, though Hive databases show up as workspaces in Drill. Each hive instance includes a default workspace that points to the Hive metastore. When you query files and tables in the hive default workspaces, you can omit the workspace name from the query.

For example, you can issue a query on a Hive table in the default workspace -using either of the following formats and get the same results:

+using either of the following queries and get the same results:

Example

SELECT * FROM hive.customers LIMIT 10;
@@ -1040,13 +1038,10 @@ SELECT * FROM hive.`default`.customers LIMIT 10;
   Default is a reserved word. You must enclose reserved words in back ticks.


 
-Because HBase instances do not have workspaces, you can use the following
-format to query a table in HBase:
+Because the HBase storage plugin configuration does not have a workspace, you can use the following
+query:
 SELECT * FROM hbase.customers LIMIT 10;
 
-After you register a data source as a storage plugin instance with Drill, and
-optionally configure workspaces, you can query the data source.
-
     
       
         

http://git-wip-us.apache.org/repos/asf/drill-site/blob/d0ba86f3/feed.xml
----------------------------------------------------------------------
diff --git a/feed.xml b/feed.xml
index 3e6c72f..8e3f4ea 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
 
     /
     
-    Tue, 07 Jul 2015 18:15:20 -0700
-    Tue, 07 Jul 2015 18:15:20 -0700
+    Thu, 16 Jul 2015 18:16:54 -0700
+    Thu, 16 Jul 2015 18:16:54 -0700
     Jekyll v2.5.2