drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject svn commit: r1668106 - in /drill/site/trunk/content/drill: ./ docs/ docs/data-sources-and-file-formats/ docs/data-types/ docs/img/ docs/json-data-model/ docs/lexical-structure/ docs/sql-extensions/
Date Fri, 20 Mar 2015 17:59:10 GMT
Author: bridgetb
Date: Fri Mar 20 17:59:09 2015
New Revision: 1668106

URL: http://svn.apache.org/r1668106
Log:
DRILL-2403: add sql extensions topic to docs

Added:
    drill/site/trunk/content/drill/docs/sql-extensions/
    drill/site/trunk/content/drill/docs/sql-extensions/index.html
Modified:
    drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html
    drill/site/trunk/content/drill/docs/data-types/index.html
    drill/site/trunk/content/drill/docs/img/datasources-json-bracket.png
    drill/site/trunk/content/drill/docs/index.html
    drill/site/trunk/content/drill/docs/json-data-model/index.html
    drill/site/trunk/content/drill/docs/lexical-structure/index.html
    drill/site/trunk/content/drill/feed.xml

Modified: drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html (original)
+++ drill/site/trunk/content/drill/docs/data-sources-and-file-formats/index.html Fri Mar 20
17:59:09 2015
@@ -70,7 +70,7 @@
 <div class="int_text" align="left"><p>Included in the data sources that  Drill
supports are these key data sources:</p>
 
 <ul>
-<li>Hbase</li>
+<li>HBase</li>
 <li>Hive</li>
 <li>MapR-DB</li>
 <li>File system</li>

Modified: drill/site/trunk/content/drill/docs/data-types/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/data-types/index.html?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/data-types/index.html (original)
+++ drill/site/trunk/content/drill/docs/data-types/index.html Fri Mar 20 17:59:09 2015
@@ -534,7 +534,9 @@ c_row | c_int
 
 <h3 id="using-convert_to-and-convert_from">Using CONVERT_TO and CONVERT_FROM</h3>
 
-<p>To query HBase data in Drill, convert every column of an HBase table to/from byte
arrays from/to an <a href="/docs/data-types/">SQL data type</a> that Drill supports
when writing/reading data. For examples of how to use these functions, see <a href="/docs/sql-functions#convert-and-cast-functions">&quot;Convert
and Cast Functions&quot;.</a></p>
+<p>CONVERT_TO converts an SQL data type to complex types, including Hbase byte arrays,
JSON and Parquet arrays and mapsTo query HBase data in Drill, convert every column of an HBase
table to/from byte arrays from/to an <a href="/docs/data-types/">SQL data type</a>
that Drill supports when writing/reading data. For examples of how to use these functions,
see <a href="/docs/sql-functions#convert-and-cast-functions">&quot;Convert and Cast
Functions&quot;.</a></p>
+
+<p>CONVERT_FROM converts from complex types, including Hbase byte arrays, JSON and
Parquet arrays and maps to an SQL data type.</p>
 
 <h2 id="handling-textual-data">Handling Textual Data</h2>
 

Modified: drill/site/trunk/content/drill/docs/img/datasources-json-bracket.png
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/img/datasources-json-bracket.png?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
Binary files - no diff available.

Modified: drill/site/trunk/content/drill/docs/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/index.html?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/index.html (original)
+++ drill/site/trunk/content/drill/docs/index.html Fri Mar 20 17:59:09 2015
@@ -419,6 +419,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -667,6 +669,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -949,6 +953,8 @@
         
       
         
+      
+        
           <li><a href="/docs/installing-the-apache-drill-sandbox/">Installing
the Apache Drill Sandbox</a></li>
           
           
@@ -1238,6 +1244,8 @@
               
             
               
+            
+              
                 <li><a href="/docs/installing-the-mapr-sandbox-with-apache-drill-on-vmware-player-vmware-fusion/">Installing
the MapR Sandbox with Apache Drill on VMware Player/VMware Fusion</a></li>
               
             
@@ -1731,6 +1739,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -1908,6 +1918,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -2279,6 +2291,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -2525,6 +2539,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -2976,6 +2992,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -3283,6 +3301,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -3441,6 +3461,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -3927,6 +3949,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -4232,6 +4256,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -4546,6 +4572,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -4667,6 +4695,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -5170,6 +5200,8 @@
               
             
               
+            
+              
                 <li><a href="/docs/date-time-and-timestamp/">Date, Time, and
Timestamp</a></li>
               
             
@@ -5496,6 +5528,8 @@
               
             
               
+            
+              
                 <li><a href="/docs/flatten-function/">FLATTEN Function</a></li>
               
             
@@ -5773,6 +5807,8 @@
               
             
               
+            
+              
                 <li><a href="/docs/alter-session-command/">ALTER SESSION Command</a></li>
               
             
@@ -5854,6 +5890,13 @@
         
       
         
+          <li><a href="/docs/sql-extensions/">SQL Extensions</a></li>
+          
+          
+          
+        
+      
+        
       
         
       
@@ -6218,6 +6261,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -6540,6 +6585,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -7008,6 +7055,8 @@
             
               
             
+              
+            
             </ul>
           
         
@@ -7160,6 +7209,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -7472,6 +7523,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -7794,6 +7847,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -8101,6 +8156,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -8413,6 +8470,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -8735,6 +8794,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -9042,6 +9103,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -9344,6 +9407,8 @@
       
         
       
+        
+      
       </ul>
     
   
@@ -9356,6 +9421,8 @@
   
 
   
+
+  
 
   
 

Modified: drill/site/trunk/content/drill/docs/json-data-model/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/json-data-model/index.html?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/json-data-model/index.html (original)
+++ drill/site/trunk/content/drill/docs/json-data-model/index.html Fri Mar 20 17:59:09 2015
@@ -204,6 +204,10 @@ SELECT my column from dfs.`&lt;path_file
 
 <p>The first index position of an array is 0.</p>
 
+<ul>
+<li>Do not use a map, array or repeated scalar type in GROUP BY, ORDER BY or in a comparison
operator.</li>
+</ul>
+
 <p>Drill returns null when a document does not have the specified map or level.</p>
 
 <p>Using the following techniques, you can query complex, nested JSON:</p>

Modified: drill/site/trunk/content/drill/docs/lexical-structure/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/lexical-structure/index.html?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
--- drill/site/trunk/content/drill/docs/lexical-structure/index.html (original)
+++ drill/site/trunk/content/drill/docs/lexical-structure/index.html Fri Mar 20 17:59:09 2015
@@ -95,6 +95,16 @@
 
 <p>SQL function and command names are case-insensitive. Storage plugin and workspace
names are case-sensitive. Column and table names are case-insensitive unless enclosed in double
quotation marks. The double-quotation mark character can be used as an escape character for
the double quotation mark.</p>
 
+<p>Although column names are case-insensitive in Drill, the names might be otherwise
in the storage format:</p>
+
+<ul>
+<li>JSON: insensitive</li>
+<li>Hive: insensitive</li>
+<li>Parquet: insensitive</li>
+<li>MapR-DB: case-sensitive</li>
+<li>HBase: case-sensitive</li>
+</ul>
+
 <p>Keywords are case-insensitive. For example, the keywords SELECT and select are equivalent.
This document shows keywords in uppercase.</p>
 
 <p>The sys.options table name and values are case-sensitive. The following query works:</p>

Added: drill/site/trunk/content/drill/docs/sql-extensions/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/sql-extensions/index.html?rev=1668106&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/sql-extensions/index.html (added)
+++ drill/site/trunk/content/drill/docs/sql-extensions/index.html Fri Mar 20 17:59:09 2015
@@ -0,0 +1,224 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>SQL Extensions - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes"
target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue
Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill"
target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding:
0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>SQL Extensions</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>Drill extends SQL to work with Hadoop-scale
data and to explore smaller-scale data in ways not possible with SQL. Using intuitive SQL
extensions you work with self-describing data and complex data types. Extensions to SQL include
capabilities for exploring self-describing data, such as files and HBase, directly in the
native format.</p>
+
+<p>Drill provides language support for pointing to <a href="">storage plugin</a>
interfaces that Drill uses to interact with data sources. Use the name of a storage plugin
to specify a file system <em>database</em> as a prefix in queries when you refer
to objects across databases. Query files, including compressed .gz files and directories like
an SQL table using a single query. </p>
+
+<p>Drill extends the SELECT statement for reading complex, multi-structured data. The
extended CREATE TABLE AS SELECT, provides the capability to write data of complex/multi-structured
data types. Drill extends the <a href="http://drill.apache.org/docs/lexical-structure">lexical
rules</a> for working with files and directories, such as using back ticks for including
file names, directory names, and reserved words in queries. Drill syntax supports using the
file system as a persistent store for query profiles and diagnostic information.</p>
+
+<h2 id="extensions-for-hive--and-hbase-related-data-sources">Extensions for Hive- and
HBase-related Data Sources</h2>
+
+<p>Drill supports Hive and HBase as a plug-and-play data source. You can query Hive
tables with no modifications and creating model in the Hive metastore. Primitives, such as
JOIN, support columnar operation.</p>
+
+<h2 id="extensions-for-json-related-data-sources">Extensions for JSON-related Data
Sources</h2>
+
+<p>For reading all JSON data as text, use the all text mode extension. Drill extends
SQL to provide access to repeating values in arrays and arrays within arrays (array indexes).
You can use these extensions to reach into deeply nested data. Drill extensions use standard
JavaScript notation for referencing data elements in a hierarchy.</p>
+
+<h2 id="extensions-for-text-data-sources">Extensions for Text Data Sources</h2>
+
+<p>Drill handles plain text files and directories like standard SQL tables and can
infer knowledge about the schema of the data. You can query compressed .gz files.</p>
+
+<h2 id="sql-commands-extensions">SQL Commands Extensions</h2>
+
+<p>The following table describes key Drill extensions to SQL commands.</p>
+
+<table>
+  <tr>
+    <th>Command</th>
+    <th>SQL</th>
+    <th>Drill</th>
+  </tr>
+  <tr>
+    <td>ALTER (SESSION | SYSTEM)</td>
+    <td>None</td>
+    <td>Changes a system or session option.</td>
+  </tr>
+  <tr>
+    <td>CREATE TABLE AS SELECT</td>
+    <td>Creates a table from selected data in an existing database table.</td>
+    <td>Stores selected data from one or more data sources on the file system.</td>
+  </tr>
+  <tr>
+    <td>CREATE VIEW</td>
+    <td>Creates a virtual table. The fields in a view are fields from one or more real
tables in the database.</td>
+    <td>Creates a virtual structure for and stores the result set. The fields in a
view are fields from files in a file system, Hive, Hbase, MapR-DB tables</td>
+  </tr>
+  <tr>
+    <td>DESCRIBE</td>
+    <td>Obtains information about the &lt;select list&gt; columns</td>
+    <td>Obtains information about views created in a workspace and tables created in
Hive, HBase, and MapR-DB.</td>
+  </tr>
+  <tr>
+    <td>EXPLAIN</td>
+    <td>None</td>
+    <td>Obtains a query execution plan.</td>
+  </tr>
+  <tr>
+    <td>INSERT</td>
+    <td>Loads data into the database for querying.</td>
+    <td>No INSERT function. Performs schema-on-read querying and execution; no need
to load data into Drill for querying.</td>
+  </tr>
+  <tr>
+    <td>SELECT</td>
+    <td>Retrieves rows from a database table or view.</td>
+    <td>Retrieves data from Hbase, Hive, MapR-DB, file system or other storage plugin
data source.</td>
+  </tr>
+  <tr>
+    <td>SHOW (DATABASES | SCHEMAS | FILES | TABLES)</td>
+    <td>None</td>
+    <td>Lists the storage plugin data sources available for querying or the Hive, Hbase,
MapR-DB tables, or views for the data source in use. Supports a FROM clause for listing file
data sources in directories.</td>
+  </tr>
+  <tr>
+    <td>USE</td>
+    <td>Targets a database in SQL schema for querying.</td>
+    <td>Targets Hbase, Hive, MapR-DB, file system or other storage plugin data source,
which can be schema-less for querying.</td>
+  </tr>
+</table>
+
+<h2 id="sql-function-extensions">SQL Function Extensions</h2>
+
+<p>The following table describes key Drill functions for analyzing nested data.</p>
+
+<table>
+  <tr>
+    <th>Function</th>
+    <th>SQL</th>
+    <th>Drill</th>
+  </tr>
+  <tr>
+    <td>CAST</td>
+    <td>Casts database data from one type to another.</td>
+    <td>Casts database data from one type to another and also casts data having no
metadata into a readable type. Allows liberal casting of schema-less data.</td>
+  </tr>
+  <tr>
+    <td>CONVERT_TO</td>
+    <td>Converts an expression from one type to another using the CONVERT command.</td>
+    <td>Converts an SQL data type to complex types, including Hbase byte arrays, JSON
and Parquet arrays and maps.</td>
+  </tr>
+  <tr>
+    <td>CONVERT_FROM</td>
+    <td>Same as above</td>
+    <td>Converts from complex types, including Hbase byte arrays, JSON and Parquet
arrays and maps to an SQL data type.</td>
+  </tr>
+  <tr>
+    <td>FLATTEN</td>
+    <td>None</td>
+    <td>Separates the elements in nested data from a repeated field into individual
records.</td>
+  </tr>
+  <tr>
+    <td>KVGEN</td>
+    <td>None</td>
+    <td>Returns a repeated map, generating key-value pairs to simplify querying of
complex data having unknown column names. You can then aggregate or filter on the key or value.</td>
+  </tr>
+  <tr>
+    <td>REPEATED_COUNT</td>
+    <td>None</td>
+    <td>Counts the values in a JSON array.</td>
+  </tr>
+</table>
+
+<h2 id="other-extensions">Other Extensions</h2>
+
+<p><a href=""><code>sys</code> database system tables</a> provide
port, version, and option information. Drill Connects to a random node, know where you’re
connected:</p>
+
+<p>select host from sys.drillbits where <code>current</code> = true;
++------------+
+|    host    |
++------------+
+| 10.1.1.109 |
++------------+</p>
+
+<p>select commit_id from sys.version;
++------------+
+| commit_id  |
++------------+
+| e3ab2c1760ad34bda80141e2c3108f7eda7c9104 |</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License,
Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other
names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Modified: drill/site/trunk/content/drill/feed.xml
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/feed.xml?rev=1668106&r1=1668105&r2=1668106&view=diff
==============================================================================
--- drill/site/trunk/content/drill/feed.xml (original)
+++ drill/site/trunk/content/drill/feed.xml Fri Mar 20 17:59:09 2015
@@ -6,8 +6,8 @@
 </description>
     <link>/</link>
     <atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Wed, 18 Mar 2015 13:32:28 -0700</pubDate>
-    <lastBuildDate>Wed, 18 Mar 2015 13:32:28 -0700</lastBuildDate>
+    <pubDate>Fri, 20 Mar 2015 10:55:36 -0700</pubDate>
+    <lastBuildDate>Fri, 20 Mar 2015 10:55:36 -0700</lastBuildDate>
     <generator>Jekyll v2.5.2</generator>
     
       <item>



Mime
View raw message