drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tshi...@apache.org
Subject svn commit: r1651949 [8/13] - in /drill/site/trunk/content/drill: ./ blog/2014/11/19/sql-on-mongodb/ blog/2014/12/02/drill-top-level-project/ blog/2014/12/09/running-sql-queries-on-amazon-s3/ blog/2014/12/11/apache-drill-qa-panelist-spotlight/ blog/201...
Date Thu, 15 Jan 2015 05:11:48 GMT
Added: drill/site/trunk/content/drill/docs/lession-3-run-queries-on-complex-data-types/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/lession-3-run-queries-on-complex-data-types/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/lession-3-run-queries-on-complex-data-types/index.html (added)
+++ drill/site/trunk/content/drill/docs/lession-3-run-queries-on-complex-data-types/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,444 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Lession 3: Run Queries on Complex Data Types - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Lession 3: Run Queries on Complex Data Types</h1>
+
+</div>
+
+<div class="int_text" align="left"><h2 id="goal">Goal</h2>
+
+<p>This lesson focuses on queries that exercise functions and operators on self-
+describing data and complex data types. Drill offers intuitive SQL extensions
+to work with such data and offers high query performance with an architecture
+built from the ground up for complex data.</p>
+
+<h2 id="queries-in-this-lesson">Queries in This Lesson</h2>
+
+<p>Now that you have run ANSI SQL queries against different tables and files with
+relational data, you can try some examples including complex types.</p>
+
+<ul>
+<li>Access directories and subdirectories of files in a single SELECT statement.</li>
+<li>Demonstrate simple ways to access complex data in JSON files.</li>
+<li>Demonstrate the repeated_count function to aggregate values in an array.</li>
+</ul>
+
+<h2 id="query-partitioned-directories">Query Partitioned Directories</h2>
+
+<p>You can use special variables in Drill to refer to subdirectories in your
+workspace path:</p>
+
+<ul>
+<li>dir0</li>
+<li>dir1</li>
+<li>…</li>
+</ul>
+
+<p>Note that these variables are dynamically determined based on the partitioning
+of the file system. No up-front definitions are required on what partitions
+exist. Here is a visual example of how this works:</p>
+
+<p><img src="../../img/example_query.png" alt="example_query.png"></p>
+
+<h3 id="set-workspace-to-dfs.logs:">Set workspace to dfs.logs:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; use dfs.logs;
++------------+------------+
+| ok | summary |
++------------+------------+
+| true | Default schema changed to &#39;dfs.logs&#39; |
++------------+------------+
+</code></pre></div>
+<h3 id="query-logs-data-for-a-specific-year:">Query logs data for a specific year:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select * from logs where dir0=&#39;2013&#39; limit 10;
++------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-----------+------------+
+| dir0 | dir1 | trans_id | date | time | cust_id | device | state | camp_id | keywords | prod_id | purch_flag |
++------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-----------+------------+
+| 2013 | 11 | 12119 | 11/09/2013 | 02:24:51 | 262 | IOS5 | ny | 0 | chamber | 198 | false |
+| 2013 | 11 | 12120 | 11/19/2013 | 09:37:43 | 0 | AOS4.4 | il | 2 | outside | 511 | false |
+| 2013 | 11 | 12134 | 11/10/2013 | 23:42:47 | 60343 | IOS5 | ma | 4 | and | 421 | false |
+| 2013 | 11 | 12135 | 11/16/2013 | 01:42:13 | 46762 | AOS4.3 | ca | 4 | here&#39;s | 349 | false |
+| 2013 | 11 | 12165 | 11/26/2013 | 21:58:09 | 41987 | AOS4.2 | mn | 4 | he | 271 | false |
+| 2013 | 11 | 12168 | 11/09/2013 | 23:41:48 | 8600 | IOS5 | in | 6 | i | 459 | false |
+| 2013 | 11 | 12196 | 11/20/2013 | 02:23:06 | 15603 | IOS5 | tn | 1 | like | 324 | false |
+| 2013 | 11 | 12203 | 11/25/2013 | 23:50:29 | 221 | IOS6 | tx | 10 | if | 323 | false |
+| 2013 | 11 | 12206 | 11/09/2013 | 23:53:01 | 2488 | AOS4.2 | tx | 14 | unlike | 296 | false |
+| 2013 | 11 | 12217 | 11/06/2013 | 23:51:56 | 0 | AOS4.2 | tx | 9 | can&#39;t | 54 | false |
++------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+
+</code></pre></div>
+<p>This query constrains files inside the subdirectory named 2013. The variable
+dir0 refers to the first level down from logs, dir1 to the next level, and so
+on. So this query returned 10 of the rows for February 2013.</p>
+
+<h3 id="further-constrain-the-results-using-multiple-predicates-in-the-query:">Further constrain the results using multiple predicates in the query:</h3>
+
+<p>This query returns a list of customer IDs for people who made a purchase via
+an IOS5 device in August 2013.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select dir0 as yr, dir1 as mth, cust_id from logs
+where dir0=&#39;2013&#39; and dir1=&#39;8&#39; and device=&#39;IOS5&#39; and purch_flag=&#39;true&#39;
+order by `date`;
++------------+------------+------------+
+| yr | mth | cust_id |
++------------+------------+------------+
+| 2013 | 8 | 4 |
+| 2013 | 8 | 521 |
+| 2013 | 8 | 1 |
+| 2013 | 8 | 2 |
+| 2013 | 8 | 4 |
+| 2013 | 8 | 549 |
+| 2013 | 8 | 72827 |
+| 2013 | 8 | 38127 |
+...
+</code></pre></div>
+<h3 id="return-monthly-counts-per-customer-for-a-given-year:">Return monthly counts per customer for a given year:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select cust_id, dir1 month_no, count(*) month_count from logs
+where dir0=2014 group by cust_id, dir1 order by cust_id, month_no limit 10;
++------------+------------+-------------+
+|  cust_id   |  month_no  | month_count |
++------------+------------+-------------+
+| 0          | 1          | 143         |
+| 0          | 2          | 118         |
+| 0          | 3          | 117         |
+| 0          | 4          | 115         |
+| 0          | 5          | 137         |
+| 0          | 6          | 117         |
+| 0          | 7          | 142         |
+| 0          | 8          | 19          |
+| 1          | 1          | 66          |
+| 1          | 2          | 59          |
++------------+------------+-------------+
+10 rows selected
+</code></pre></div>
+<p>This query groups the aggregate function by customer ID and month for one
+year: 2014.</p>
+
+<h2 id="query-complex-data">Query Complex Data</h2>
+
+<p>Drill provides some specialized operators and functions that you can use to
+analyze nested data natively without transformation. If you are familiar with
+JavaScript notation, you will already know how some of these extensions work.</p>
+
+<h3 id="set-the-workspace-to-dfs.clicks:">Set the workspace to dfs.clicks:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; use dfs.clicks;
++------------+------------+
+| ok | summary |
++------------+------------+
+| true | Default schema changed to &#39;dfs.clicks&#39; |
++------------+------------+
+</code></pre></div>
+<h3 id="explore-clickstream-data:">Explore clickstream data:</h3>
+
+<p>Note that the user_info and trans_info columns contain nested data: arrays and
+arrays within arrays. The following queries show how to access this complex
+data.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select * from `clicks/clicks.json` limit 5;
++------------+------------+------------+------------+------------+
+| trans_id | date | time | user_info | trans_info |
++------------+------------+------------+------------+------------+
+| 31920 | 2014-04-26 | 12:17:12 | {&quot;cust_id&quot;:22526,&quot;device&quot;:&quot;IOS5&quot;,&quot;state&quot;:&quot;il&quot;} | {&quot;prod_id&quot;:[174,2],&quot;purch_flag&quot;:&quot;false&quot;} |
+| 31026 | 2014-04-20 | 13:50:29 | {&quot;cust_id&quot;:16368,&quot;device&quot;:&quot;AOS4.2&quot;,&quot;state&quot;:&quot;nc&quot;} | {&quot;prod_id&quot;:[],&quot;purch_flag&quot;:&quot;false&quot;} |
+| 33848 | 2014-04-10 | 04:44:42 | {&quot;cust_id&quot;:21449,&quot;device&quot;:&quot;IOS6&quot;,&quot;state&quot;:&quot;oh&quot;} | {&quot;prod_id&quot;:[582],&quot;purch_flag&quot;:&quot;false&quot;} |
+| 32383 | 2014-04-18 | 06:27:47 | {&quot;cust_id&quot;:20323,&quot;device&quot;:&quot;IOS5&quot;,&quot;state&quot;:&quot;oh&quot;} | {&quot;prod_id&quot;:[710,47],&quot;purch_flag&quot;:&quot;false&quot;} |
+| 32359 | 2014-04-19 | 23:13:25 | {&quot;cust_id&quot;:15360,&quot;device&quot;:&quot;IOS5&quot;,&quot;state&quot;:&quot;ca&quot;} | {&quot;prod_id&quot;: [0,8,170,173,1,124,46,764,30,711,0,3,25],&quot;purch_flag&quot;:&quot;true&quot;} |
++------------+------------+------------+------------+------------+
+</code></pre></div>
+<h3 id="unpack-the-user_info-column:">Unpack the user_info column:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select t.user_info.cust_id as custid, t.user_info.device as device,
+t.user_info.state as state
+from `clicks/clicks.json` t limit 5;
++------------+------------+------------+
+| custid | device | state |
++------------+------------+------------+
+| 22526 | IOS5 | il |
+| 16368 | AOS4.2 | nc |
+| 21449 | IOS6 | oh |
+| 20323 | IOS5 | oh |
+| 15360 | IOS5 | ca |
++------------+------------+------------+
+</code></pre></div>
+<p>This query uses a simple table.column.column notation to extract nested column
+data. For example:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">t.user_info.cust_id
+</code></pre></div>
+<p>where <code>t</code> is the table alias provided in the query, <code>user_info</code> is a top-level
+column name, and <code>cust_id</code> is a nested column name.</p>
+
+<p>The table alias is required; otherwise column names such as <code>user_info</code> are
+parsed as table names by the SQL parser.</p>
+
+<h3 id="unpack-the-trans_info-column:">Unpack the trans_info column:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select t.trans_info.prod_id as prodid, t.trans_info.purch_flag as
+purchased
+from `clicks/clicks.json` t limit 5;
++------------+------------+
+| prodid | purchased |
++------------+------------+
+| [174,2] | false |
+| [] | false |
+| [582] | false |
+| [710,47] | false |
+| [0,8,170,173,1,124,46,764,30,711,0,3,25] | true |
++------------+------------+
+5 rows selected
+</code></pre></div>
+<p>Note that this result reveals that the prod_id column contains an array of IDs
+(one or more product ID values per row, separated by commas). The next step
+shows how you to access this kind of data.</p>
+
+<h2 id="query-arrays">Query Arrays</h2>
+
+<p>Now use the [ n ] notation, where n is the position of the value in an array,
+starting from position 0 (not 1) for the first value. You can use this
+notation to write interesting queries against nested array data.</p>
+
+<p>For example:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">trans_info.prod_id[0]
+</code></pre></div>
+<p>refers to the first value in the nested prod_id column and</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">trans_info.prod_id[20]
+</code></pre></div>
+<p>refers to the 21st value, assuming one exists.</p>
+
+<h3 id="find-the-first-product-that-is-searched-for-in-each-transaction:">Find the first product that is searched for in each transaction:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select t.trans_id, t.trans_info.prod_id[0] from `clicks/clicks.json` t limit 5;
++------------+------------+
+|  trans_id  |   EXPR$1   |
++------------+------------+
+| 31920      | 174        |
+| 31026      | null       |
+| 33848      | 582        |
+| 32383      | 710        |
+| 32359      | 0          |
++------------+------------+
+5 rows selected
+</code></pre></div>
+<h3 id="for-which-transactions-did-customers-search-on-at-least-21-products?">For which transactions did customers search on at least 21 products?</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select t.trans_id, t.trans_info.prod_id[20]
+from `clicks/clicks.json` t
+where t.trans_info.prod_id[20] is not null
+order by trans_id limit 5;
++------------+------------+
+|  trans_id  |   EXPR$1   |
++------------+------------+
+| 10328      | 0          |
+| 10380      | 23         |
+| 10701      | 1          |
+| 11100      | 0          |
+| 11219      | 46         |
++------------+------------+
+5 rows selected
+</code></pre></div>
+<p>This query returns transaction IDs and product IDs for records that contain a
+non-null product ID at the 21st position in the array.</p>
+
+<h3 id="return-clicks-for-a-specific-product-range:">Return clicks for a specific product range:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select * from (select t.trans_id, t.trans_info.prod_id[0] as prodid,
+t.trans_info.purch_flag as purchased
+from `clicks/clicks.json` t) sq
+where sq.prodid between 700 and 750 and sq.purchased=&#39;true&#39;
+order by sq.prodid;
++------------+------------+------------+
+| trans_id | prodid | purchased |
++------------+------------+------------+
+| 21886 | 704 | true |
+| 20674 | 708 | true |
+| 22158 | 709 | true |
+| 34089 | 714 | true |
+| 22545 | 714 | true |
+| 37500 | 717 | true |
+| 36595 | 718 | true |
+...
+</code></pre></div>
+<p>This query assumes that there is some meaning to the array (that it is an
+ordered list of products purchased rather than a random list).</p>
+
+<h2 id="perform-operations-on-arrays">Perform Operations on Arrays</h2>
+
+<h3 id="rank-successful-click-conversions-and-count-product-searches-for-each-session:">Rank successful click conversions and count product searches for each session:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select t.trans_id, t.`date` as session_date, t.user_info.cust_id as
+cust_id, t.user_info.device as device, repeated_count(t.trans_info.prod_id) as
+prod_count, t.trans_info.purch_flag as purch_flag
+from `clicks/clicks.json` t
+where t.trans_info.purch_flag = &#39;true&#39; order by prod_count desc;
++------------+--------------+------------+------------+------------+------------+
+| trans_id | session_date | cust_id | device | prod_count | purch_flag |
++------------+--------------+------------+------------+------------+------------+
+| 37426 | 2014-04-06 | 18709 | IOS5 | 34 | true |
+| 31589 | 2014-04-16 | 18576 | IOS6 | 31 | true |
+| 11600 | 2014-04-07 | 4260 | AOS4.2 | 28 | true |
+| 35074 | 2014-04-03 | 16697 | AOS4.3 | 27 | true |
+| 17192 | 2014-04-22 | 2501 | AOS4.2 | 26 | true |
+...
+</code></pre></div>
+<p>This query uses a Drill SQL extension, the repeated_count function, to get an
+aggregated count of the array values. The query returns the number of products
+searched for each session that converted into a purchase and ranks the counts
+in descending order. Only clicks that have resulted in a purchase are counted.</p>
+
+<h2 id="store-a-result-set-in-a-table-for-reuse-and-analysis">Store a Result Set in a Table for Reuse and Analysis</h2>
+
+<p>Finally, run another correlated subquery that returns a fairly large result
+set. To facilitate additional analysis on this result set, you can easily and
+quickly create a Drill table from the results of the query.</p>
+
+<h3 id="continue-to-use-the-dfs.clicks-workspace">Continue to use the dfs.clicks workspace</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; use dfs.clicks;
++------------+------------+
+| ok | summary |
++------------+------------+
+| true | Default schema changed to &#39;dfs.clicks&#39; |
++------------+------------+
+</code></pre></div>
+<h3 id="return-product-searches-for-high-value-customers:">Return product searches for high-value customers:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select o.cust_id, o.order_total, t.trans_info.prod_id[0] as prod_id 
+from hive.orders as o, `clicks/clicks.json` t 
+where o.cust_id=t.user_info.cust_id 
+and o.order_total &gt; (select avg(inord.order_total) 
+from hive.orders inord where inord.state = o.state);
++------------+-------------+------------+
+|  cust_id   | order_total |   prod_id   |
++------------+-------------+------------+
+...
+| 9650       | 69          | 16         |
+| 9650       | 69          | 560        |
+| 9650       | 69          | 959        |
+| 9654       | 76          | 768        |
+| 9656       | 76          | 32         |
+| 9656       | 76          | 16         |
+...
++------------+-------------+------------+
+106,281 rows selected
+</code></pre></div>
+<p>This query returns a list of products that are being searched for by customers
+who have made transactions that are above the average in their states.</p>
+
+<h3 id="materialize-the-result-of-the-previous-query:">Materialize the result of the previous query:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; create table product_search as select o.cust_id, o.order_total, t.trans_info.prod_id[0] as prod_id
+from hive.orders as o, `clicks/clicks.json` t 
+where o.cust_id=t.user_info.cust_id and o.order_total &gt; (select avg(inord.order_total) 
+from hive.orders inord where inord.state = o.state);
++------------+---------------------------+
+|  Fragment  | Number of records written |
++------------+---------------------------+
+| 0_0        | 106281                    |
++------------+---------------------------+
+1 row selected
+</code></pre></div>
+<p>This example uses a CTAS statement to create a table based on a correlated
+subquery that you ran previously. This table contains all of the rows that the
+query returns (106,281) and stores them in the format specified by the storage
+plugin (Parquet format in this example). You can create tables that store data
+in csv, parquet, and json formats.</p>
+
+<h3 id="query-the-new-table-to-verify-the-row-count:">Query the new table to verify the row count:</h3>
+
+<p>This example simply checks that the CTAS statement worked by verifying the
+number of rows in the table.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:&gt; select count(*) from product_search;
++------------+
+|   EXPR$0   |
++------------+
+| 106281     |
++------------+
+1 row selected
+</code></pre></div>
+<h3 id="find-the-storage-file-for-the-table:">Find the storage file for the table:</h3>
+<div class="highlight"><pre><code class="language-text" data-lang="text">[root@maprdemo product_search]# cd /mapr/demo.mapr.com/data/nested/product_search
+[root@maprdemo product_search]# ls -la
+total 451
+drwxr-xr-x. 2 mapr mapr      1 Sep 15 13:41 .
+drwxr-xr-x. 4 root root      2 Sep 15 13:41 ..
+-rwxr-xr-x. 1 mapr mapr 460715 Sep 15 13:41 0_0_0.parquet
+</code></pre></div>
+<p>Note that the table is stored in a file called <code>0_0_0.parquet</code>. This file is
+stored in the location defined by the dfs.clicks workspace:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">&quot;location&quot;: &quot;http://demo.mapr.com/data/nested)&quot;
+</code></pre></div>
+<p>with a subdirectory that has the same name as the table you created.</p>
+
+<h2 id="what&#39;s-next">What&#39;s Next</h2>
+
+<p>Complete the tutorial with the <a href="/confluence/display/DRILL/Summary">Summary</a>.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/manage-drill/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/manage-drill/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/manage-drill/index.html (added)
+++ drill/site/trunk/content/drill/docs/manage-drill/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,110 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Manage Drill - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Manage Drill</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>When using Drill, you may need to stop and restart a Drillbit on a node, or
+modify various options. For example, the default storage format for CTAS
+statements is Parquet. You can modify the default setting so that output data
+is stored in CSV or JSON format.</p>
+
+<p>You can use certain SQL commands to manage Drill from within the Drill shell
+(SQLLine). You can also modify Drill configuration options, such as memory
+allocation, in Drill&#39;s configuration files.</p>
+
+<p>Refer to the following documentation for information about managing Drill in
+your cluster:</p>
+
+<ul>
+<li><a href="/confluence/display/DRILL/Configuration+Options">Configuration Options</a></li>
+<li><a href="/confluence/pages/viewpage.action?pageId=44994063">Starting/Stopping Drill</a></li>
+<li><a href="/confluence/display/DRILL/Ports+Used+by+Drill">Ports Used by Drill</a></li>
+<li><a href="/confluence/display/DRILL/Partition+Pruning">Partition Pruning</a></li>
+<li><a href="/confluence/display/DRILL/Monitoring+and+Canceling+Queries+in+the+Drill+Web+UI">Monitoring and Canceling Queries in the Drill Web UI</a></li>
+</ul>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/mapr-db-plugin-for-apache-drill/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/mapr-db-plugin-for-apache-drill/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/mapr-db-plugin-for-apache-drill/index.html (added)
+++ drill/site/trunk/content/drill/docs/mapr-db-plugin-for-apache-drill/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,118 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>MapR-DB Plugin for Apache Drill - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>MapR-DB Plugin for Apache Drill</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>Drill includes a <code>maprdb</code> format plugin for MapR-DB that is defined within the
+default <code>dfs</code> storage plugin instance when you install Drill from the <code>mapr-
+drill</code> package on a MapR node. The <code>maprdb</code> format plugin improves the
+estimated number of rows that Drill uses to plan a query. It also enables you
+to query tables like you would query files in a file system because MapR-DB
+and MapR-FS share the same namespace.</p>
+
+<p>You can query tables stored across multiple directories. You do not need to
+create a table mapping to a directory before you query a table in the
+directory. You can select from any table in any directory the same way you
+would select from files in MapR-FS, using the same syntax.</p>
+
+<p>Instead of including the name of a file, you include the table name in the
+query.</p>
+
+<p><strong>Example</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT * FROM mfs.`/users/max/mytable`;
+</code></pre></div>
+<p>Drill stores the <code>maprdb</code> format plugin in the <code>dfs</code> storage plugin instance,
+which you can view in the Drill Web UI. You can access the Web UI at
+<a href="http://localhost:8047/storage">http://localhost:8047/storage</a>. Click <strong>Update</strong> next to the <code>dfs</code> instance
+in the Web UI to view the configuration for the <code>dfs</code> instance.</p>
+
+<p>The following image shows a portion of the configuration with the <code>maprdb</code>
+format plugin for the<code>dfs</code> instance:</p>
+
+<p><img src="../../img/18.png" alt=""></p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/memory-allocation/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/memory-allocation/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/memory-allocation/index.html (added)
+++ drill/site/trunk/content/drill/docs/memory-allocation/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,118 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Memory Allocation - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Memory Allocation</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>You can configure the amount of direct memory allocated to a Drillbit for
+query processing. The default limit is 8G, but Drill prefers 16G or more
+depending on the workload. The total amount of direct memory that a Drillbit
+allocates to query operations cannot exceed the limit set.</p>
+
+<p>Drill mainly uses Java direct memory and performs well when executing
+operations in memory instead of storing the operations on disk. Drill does not
+write to disk unless absolutely necessary, unlike MapReduce where everything
+is written to disk during each phase of a job.</p>
+
+<p>The JVM’s heap memory does not limit the amount of direct memory available in
+a Drillbit. The on-heap memory for Drill is only about 4-8G, which should
+suffice because Drill avoids having data sit in heap memory.</p>
+
+<h4 id="modifying-drillbit-memory">Modifying Drillbit Memory</h4>
+
+<p>You can modify memory for each Drillbit node in your cluster. To modify the
+memory for a Drillbit, edit the <code>XX:MaxDirectMemorySize</code> parameter in the
+Drillbit startup script located in <code>&lt;drill_installation_directory&gt;/conf/drill-
+env.sh</code>.</p>
+
+<p><strong>Note:</strong> If this parameter is not set, the limit depends on the amount of available system memory.</p>
+
+<p>After you edit <code>&lt;drill_installation_directory&gt;/conf/drill-env.sh</code>, <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=44994063">restart
+the Drillbit
+</a>on
+the node.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/mongodb-plugin-for-apache-drill/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/mongodb-plugin-for-apache-drill/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/mongodb-plugin-for-apache-drill/index.html (added)
+++ drill/site/trunk/content/drill/docs/mongodb-plugin-for-apache-drill/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,252 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>MongoDB Plugin for Apache Drill - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>MongoDB Plugin for Apache Drill</h1>
+
+</div>
+
+<div class="int_text" align="left"><h1 id="overview">Overview</h1>
+
+<p>You can leverage the power of Apache Drill to query data without any upfront
+schema definitions. Drill enables you to create an architecture that works
+with nested and dynamic schemas, making it the perfect SQL query tool to use
+on NoSQL databases, such as MongoDB.</p>
+
+<p>As of Apache Drill 0.6, you can configure MongoDB as a Drill data source.
+Drill provides a mongodb format plugin to connect to MongoDB, and run queries
+on the data using ANSI SQL.</p>
+
+<p>This tutorial assumes that you have Drill installed locally (embedded mode),
+as well as MongoDB. Examples in this tutorial use zip code aggregation data
+provided by MongDB. Before You Begin provides links to download tools and data
+used throughout the tutorial.</p>
+
+<p><strong>Note:</strong> A local instance of Drill is used in this tutorial for simplicity. You can also run Drill and MongoDB together in distributed mode.</p>
+
+<h3 id="before-you-begin">Before You Begin</h3>
+
+<p>Before you can query MongoDB with Drill, you must have Drill and MongoDB
+installed on your machine. You may also want to import the MongoDB zip code
+data to run the example queries on your machine.</p>
+
+<ol>
+<li><a href="https://cwiki.apache.org/confluence/display/DRILL/Installing+Drill+in+Embedded+Mode">Install Drill</a>, if you do not already have it installed on your machine.</li>
+<li><a href="http://docs.mongodb.org/manual/installation/">Install MongoDB</a>, if you do not already have it installed on your machine.</li>
+<li><a href="http://docs.mongodb.org/manual/tutorial/aggregation-zip-code-data-set/">Import the MongoDB zip code sample data set</a>. You can use Mongo Import to get the data. </li>
+</ol>
+
+<h2 id="configuring-mongodb">Configuring MongoDB</h2>
+
+<p>Start Drill and configure the MongoDB storage plugin instance in the Drill Web
+UI to connect to Drill. Drill must be running in order to access the Web UI.</p>
+
+<p>Complete the following steps to configure MongoDB as a data source for Drill:</p>
+
+<ol>
+<li><p>Navigate to <code>&lt;drill_installation_directory&gt;/drill-&lt;version&gt;,</code> and enter the following command to invoke SQLLine and start Drill:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">bin/sqlline -u jdbc:drill:zk=local -n admin -p admin
+</code></pre></div>
+<p>When Drill starts, the following prompt appears:<br>
+<code>0: jdbc:drill:zk=local&gt;  
+</code>Do not enter any commands. You will return to the command prompt after
+completing the configuration in the Drill Web UI.</p></li>
+<li><p>Open a browser window, and navigate to the Drill Web UI at <code>http://localhost:8047</code>.</p></li>
+<li><p>In the navigation bar, click <strong>Storage</strong>.</p></li>
+<li><p>Under Disabled Storage Plugins, select <strong>Update</strong> next to the <code>mongo</code> instance if the instance exists. If the instance does not exist, create an instance for MongoDB.</p></li>
+<li><p>In the Configuration window, verify that <code>&quot;enabled&quot;</code> is set to <code>&quot;true.&quot;</code></p>
+
+<p><strong>Example</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">{
+  &quot;type&quot;: &quot;mongo&quot;,
+  &quot;connection&quot;: &quot;mongodb://localhost:27017/&quot;,
+  &quot;enabled&quot;: true
+}
+</code></pre></div>
+<p><strong>Note:</strong> 27017 is the default port for <code>mongodb</code> instances. </p></li>
+<li><p>Click <strong>Enable **to enable the instance, and save the</strong> **configuration.</p></li>
+<li><p>Navigate back to the Drill command line so you can query MongoDB.</p></li>
+</ol>
+
+<h2 id="querying-mongodb">Querying MongoDB</h2>
+
+<p>You can issue the <code>SHOW DATABASES</code>command to see a list of databases from all
+Drill data sources, including MongoDB. If you downloaded the zip codes file,
+you should see <code>mongo.zipdb</code> in the results.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:zk=local&gt; SHOW DATABASES;
++-------------+
+| SCHEMA_NAME |
++-------------+
+| dfs.default |
+| dfs.root    |
+| dfs.tmp     |
+| sys         |
+| mongo.zipdb |
+| cp.default  |
+| INFORMATION_SCHEMA |
++-------------+
+</code></pre></div>
+<p>If you want all queries that you submit to run on <code>mongo.zipdb</code>, you can issue
+the <code>USE</code> command to change schema.</p>
+
+<h3 id="example-queries">Example Queries</h3>
+
+<p>The following example queries are included for reference. However, you can use
+the SQL power of Apache Drill directly on MongoDB. For more information about,
+refer to the <a href="https://cwiki.apache.org/confluence/display/DRILL/SQL+Reference">Apache Drill SQL
+Reference</a>.</p>
+
+<p><strong>Example 1: View mongo.zipdb Dataset</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:zk=local&gt; SELECT * FROM zipcodes LIMIT 10;
++------------+
+|     *      |
++------------+
+| { &quot;city&quot; : &quot;AGAWAM&quot; , &quot;loc&quot; : [ -72.622739 , 42.070206] , &quot;pop&quot; : 15338 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;CUSHMAN&quot; , &quot;loc&quot; : [ -72.51565 , 42.377017] , &quot;pop&quot; : 36963 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;BARRE&quot; , &quot;loc&quot; : [ -72.108354 , 42.409698] , &quot;pop&quot; : 4546 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;BELCHERTOWN&quot; , &quot;loc&quot; : [ -72.410953 , 42.275103] , &quot;pop&quot; : 10579 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;BLANDFORD&quot; , &quot;loc&quot; : [ -72.936114 , 42.182949] , &quot;pop&quot; : 1240 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;BRIMFIELD&quot; , &quot;loc&quot; : [ -72.188455 , 42.116543] , &quot;pop&quot; : 3706 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;CHESTER&quot; , &quot;loc&quot; : [ -72.988761 , 42.279421] , &quot;pop&quot; : 1688 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;CHESTERFIELD&quot; , &quot;loc&quot; : [ -72.833309 , 42.38167] , &quot;pop&quot; : 177 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;CHICOPEE&quot; , &quot;loc&quot; : [ -72.607962 , 42.162046] , &quot;pop&quot; : 23396 , &quot;state&quot; : &quot;MA&quot;} |
+| { &quot;city&quot; : &quot;CHICOPEE&quot; , &quot;loc&quot; : [ -72.576142 , 42.176443] , &quot;pop&quot; : 31495 , &quot;state&quot; : &quot;MA&quot;} |
+</code></pre></div>
+<p><strong>Example 2: Aggregation</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:zk=local&gt; select state,city,avg(pop)     +------------+------------+------------+
+|   state    |    city    |   EXPR$2   |
++------------+------------+------------+
+| MA         | AGAWAM     | 15338.0    |
+| MA         | CUSHMAN    | 36963.0    |
+| MA         | BARRE      | 4546.0     |
+| MA         | BELCHERTOWN | 10579.0   |
+| MA         | BLANDFORD  | 1240.0     |
+| MA         | BRIMFIELD  | 3706.0     |
+| MA         | CHESTER    | 1688.0     |
+| MA         | CHESTERFIELD | 177.0    |
+| MA         | CHICOPEE   | 27445.5    |
+| MA         | WESTOVER AFB | 1764.0   |
++------------+------------+------------+
+</code></pre></div>
+<p><strong>Example 3: Nested Data Column Array</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">0: jdbc:drill:zk=local&gt; SELECT loc FROM zipcodes LIMIT 10;
++------------------------+
+|    loc                 |
++------------------------+
+| [-72.622739,42.070206] |
+| [-72.51565,42.377017]  |
+| [-72.108354,42.409698] |
+| [-72.410953,42.275103] |
+| [-72.936114,42.182949] |
+| [-72.188455,42.116543] |
+| [-72.988761,42.279421] |
+| [-72.833309,42.38167]  |
+| [-72.607962,42.162046] |
+| [-72.576142,42.176443] |
++------------------------+
+
+0: jdbc:drill:zk=local&gt; SELECT loc[0] FROM zipcodes LIMIT 10;
++------------+
+|   EXPR$0   |
++------------+
+| -72.622739 |
+| -72.51565  |
+| -72.108354 |
+| -72.410953 |
+| -72.936114 |
+| -72.188455 |
+| -72.988761 |
+| -72.833309 |
+| -72.607962 |
+| -72.576142 |
++------------+
+</code></pre></div>
+<h2 id="using-odbc/jdbc-drivers">Using ODBC/JDBC Drivers</h2>
+
+<p>You can leverage the power of Apache Drill to query MongoDB through standard
+BI tools, such as Tableau and SQuirreL.</p>
+
+<p>For information about Drill ODBC and JDBC drivers, refer to <a href="https://cwiki.apache.org/confluence/display/DRILL/Drill+Interfaces">Drill Interfaces</a>.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/monitoring-and-canceling-queries-in-the-drill-web-ui/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/monitoring-and-canceling-queries-in-the-drill-web-ui/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/monitoring-and-canceling-queries-in-the-drill-web-ui/index.html (added)
+++ drill/site/trunk/content/drill/docs/monitoring-and-canceling-queries-in-the-drill-web-ui/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,114 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Monitoring and Canceling Queries in the Drill Web UI - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Monitoring and Canceling Queries in the Drill Web UI</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>You can monitor and cancel queries from the Drill Web UI. To access the Drill
+Web UI, the Drillbit process must be running on the Drill node that you use to
+access the Drill Web UI.</p>
+
+<p>To monitor or cancel a query from the Drill Web UI, complete the following
+steps:</p>
+
+<ol>
+<li><p>Navigate to the Drill Web UI at <code>&lt;drill_node_ip_address&gt;:8047.</code><br>
+When you access the Drill Web UI, you see some general information about Drill
+running in your cluster, such as the nodes running the Drillbit process, the
+various ports Drill is using, and the amount of direct memory assigned to
+Drill.<br>
+<img src="../../img/7.png" alt=""></p></li>
+<li><p>Select <strong>Profiles</strong> in the toolbar. A list of running and completed queries appears. Drill assigns a query ID to each query and lists the Foreman node. The Foreman is the Drillbit node that receives the query from the client or application. The Foreman drives the entire query.<br>
+<img src="../../img/51.png" alt=""></p></li>
+<li><p>Click the <strong>Query ID</strong> for the query that you want to monitor or cancel. The Query and Planning window appears.<br>
+<img src="../../img/4.png" alt=""></p></li>
+<li><p>Select <strong>Edit Query</strong>.</p></li>
+<li><p>Click <strong>Cancel query **to cancel the</strong> **query. The following message appears:<br>
+<img src="../../img/46.png" alt=""></p></li>
+<li><p>Optionally, you can re-run the query to see a query summary in this window.</p></li>
+</ol>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/nested-data-functions/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/nested-data-functions/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/nested-data-functions/index.html (added)
+++ drill/site/trunk/content/drill/docs/nested-data-functions/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,99 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Nested Data Functions - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Nested Data Functions</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>This section contains descriptions of SQL functions that you can use to
+analyze nested data:</p>
+
+<ul>
+<li><a href="/confluence/display/DRILL/FLATTEN+Function">FLATTEN Function</a></li>
+<li><a href="/confluence/display/DRILL/KVGEN+Function">KVGEN Function</a></li>
+<li><a href="/confluence/display/DRILL/REPEATED_COUNT+Function">REPEATED_COUNT Function</a></li>
+</ul>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/operators/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/operators/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/operators/index.html (added)
+++ drill/site/trunk/content/drill/docs/operators/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,184 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Operators - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Operators</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>You can use various types of operators in your Drill queries to perform
+operations on your data.</p>
+
+<h2 id="logical-operators">Logical Operators</h2>
+
+<p>You can use the following logical operators in your Drill queries:</p>
+
+<ul>
+<li>AND</li>
+<li>BETWEEN</li>
+<li>IN</li>
+<li>LIKE</li>
+<li>NOT</li>
+<li>OR </li>
+</ul>
+
+<h2 id="comparison-operators">Comparison Operators</h2>
+
+<p>You can use the following comparison operators in your Drill queries:</p>
+
+<ul>
+<li>&lt;</li>
+<li>&gt;</li>
+<li>&lt;=</li>
+<li>&gt;=</li>
+<li>=</li>
+<li>&lt;&gt;</li>
+<li>IS NULL</li>
+<li>IS NOT NULL</li>
+<li>IS FALSE </li>
+<li>IS NOT FALSE</li>
+<li>IS TRUE </li>
+<li>IS NOT TRUE</li>
+</ul>
+
+<h2 id="pattern-matching-operators">Pattern Matching Operators</h2>
+
+<p>You can use the following pattern matching operators in your Drill queries:</p>
+
+<ul>
+<li>LIKE</li>
+<li>NOT LIKE</li>
+<li>SIMILAR TO</li>
+<li>NOT SIMILAR TO</li>
+</ul>
+
+<h2 id="math-operators">Math Operators</h2>
+
+<p>You can use the following math operators in your Drill queries:</p>
+
+<table><thead>
+<tr>
+<th><strong>Operator</strong></th>
+<th><strong>Description</strong></th>
+</tr>
+</thead><tbody>
+<tr>
+<td>+</td>
+<td>Addition</td>
+</tr>
+<tr>
+<td>-</td>
+<td>Subtraction</td>
+</tr>
+<tr>
+<td>*</td>
+<td>Multiplication</td>
+</tr>
+<tr>
+<td>/</td>
+<td>Division</td>
+</tr>
+</tbody></table>
+
+<h2 id="subquery-operators">Subquery Operators</h2>
+
+<p>You can use the following subquery operators in your Drill queries:</p>
+
+<ul>
+<li>EXISTS</li>
+<li>IN</li>
+</ul>
+
+<p>See <a href="https://cwiki.apache.org/confluence/display/DRILL/SELE%0ACT+Statements">SELECT Statements</a>.</p>
+
+<h2 id="string-operators">String Operators</h2>
+
+<p>You can use the following string operators in your Drill queries:</p>
+
+<ul>
+<li>string || string</li>
+<li>string || non-string or non-string || string</li>
+</ul>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/partition-pruning/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/partition-pruning/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/partition-pruning/index.html (added)
+++ drill/site/trunk/content/drill/docs/partition-pruning/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,156 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Partition Pruning - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Partition Pruning</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>Partition pruning is a performance optimization that limits the number of
+files and partitions that Drill reads when querying file systems and Hive
+tables. Drill only reads a subset of the files that reside in a file system or
+a subset of the partitions in a Hive table when a query matches certain filter
+criteria.</p>
+
+<p>For Drill to apply partition pruning to Hive tables, you must have created the
+tables in Hive using the <code>PARTITION BY</code> clause:</p>
+
+<p><code>CREATE TABLE &lt;table_name&gt; (&lt;column_name&gt;) PARTITION BY (&lt;column_name&gt;);</code></p>
+
+<p>When you create Hive tables using the <code>PARTITION BY</code> clause, each partition of
+data is automatically split out into different directories as data is written
+to disk. For more information about Hive partitioning, refer to the <a href="https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL/#LanguageManualDDL-PartitionedTables">Apache
+Hive wiki</a>.</p>
+
+<p>Typically, table data in a file system is organized by directories and
+subdirectories. Queries on table data may contain <code>WHERE</code> clause filters on
+specific directories.</p>
+
+<p>Drill’s query planner evaluates the filters as part of a Filter operator. If
+no partition filters are present, the underlying Scan operator reads all files
+in all directories and then sends the data to operators downstream, such as
+Filter.</p>
+
+<p>When partition filters are present, the query planner determines if it can
+push the filters down to the Scan such that the Scan only reads the
+directories that match the partition filters, thus reducing disk I/O.</p>
+
+<h2 id="partition-pruning-example">Partition Pruning Example</h2>
+
+<p>The /<code>Users/max/data/logs</code> directory in a file system contains subdirectories
+that span a few years.</p>
+
+<p>The following image shows the hierarchical structure of the <code>…/logs</code> directory
+and (sub) directories:</p>
+
+<p><img src="../../img/54.png" alt=""></p>
+
+<p>The following query requests log file data for 2013 from the <code>…/logs</code>
+directory in the file system:</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT * FROM dfs.`/Users/max/data/logs` WHERE cust_id &lt; 10 and dir0 = 2013 limit 2;
+</code></pre></div>
+<p>If you run the <code>EXPLAIN PLAN</code> command for the query, you can see that the<code>
+…/logs</code> directory is filtered by the scan operator.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">EXPLAIN PLAN FOR SELECT * FROM dfs.`/Users/max/data/logs` WHERE cust_id &lt; 10 and dir0 = 2013 limit 2;
+</code></pre></div>
+<p>The following image shows a portion of the physical plan when partition
+pruning is applied:</p>
+
+<p><img src="../../img/21.png" alt=""></p>
+
+<h2 id="filter-examples">Filter Examples</h2>
+
+<p>The following queries include examples of the types of filters eligible for
+partition pruning optimization:</p>
+
+<p><strong>Example 1: Partition filters ANDed together</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT * FROM dfs.`/Users/max/data/logs` WHERE dir0 = &#39;2014&#39; AND dir1 = &#39;1&#39;
+</code></pre></div>
+<p><strong>Example 2: Partition filter ANDed with regular column filter</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT * FROM dfs.`/Users/max/data/logs` WHERE cust_id &lt; 10 AND dir0 = 2013 limit 2;
+</code></pre></div>
+<p><strong>Example 3: Combination of AND, OR involving partition filters</strong></p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT * FROM dfs.`/Users/max/data/logs` WHERE (dir0 = &#39;2013&#39; AND dir1 = &#39;1&#39;) OR (dir0 = &#39;2014&#39; AND dir1 = &#39;2&#39;)
+</code></pre></div></div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>

Added: drill/site/trunk/content/drill/docs/performance/index.html
URL: http://svn.apache.org/viewvc/drill/site/trunk/content/drill/docs/performance/index.html?rev=1651949&view=auto
==============================================================================
--- drill/site/trunk/content/drill/docs/performance/index.html (added)
+++ drill/site/trunk/content/drill/docs/performance/index.html Thu Jan 15 05:11:44 2015
@@ -0,0 +1,142 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+
+
+<title>Performance - Apache Drill</title>
+
+<link href="/css/syntax.css" rel="stylesheet" type="text/css">
+<link href="/css/style.css" rel="stylesheet" type="text/css">
+<link href="/css/arrows.css" rel="stylesheet" type="text/css">
+<link href="/css/button.css" rel="stylesheet" type="text/css">
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+<link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script>
+<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script>
+<script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript" src="/js/script.js"></script>
+
+</head>
+
+<body onResize="resized();">
+
+<div class="bui"></div>
+
+<div id="search">
+<input type="text" placeholder="Enter search term here">
+</div>
+
+<div id="menu" class="mw">
+<ul>
+  <li class="logo"><a href="/"></a></li>
+  <li>
+    <a href="/overview/">Documentation</a>
+    <ul>
+      <li><a href="/overview/">Overview&nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+in+10+Minutes" target="_blank">Drill in 10 Minutes</a></li>
+      <li><a href="/why/">Why Drill? &nbsp;&nbsp;&nbsp;&nbsp;</a></li>
+      <li><a href="/architecture/">Architecture</a></li>
+    </ul>
+  </li>
+  <li>
+    <a href="/community/">Community</a>
+    <ul>
+      <li><a href="/team/">Team</a></li>
+      <li><a href="/community/#events">Events and Meetups</a></li>
+      <li><a href="/community/#mailinglists">Mailing Lists</a></li>
+      <li><a href="/community/#getinvolved">Get Involved</a></li>
+      <li><a href="https://issues.apache.org/jira/browse/DRILL/" target="_blank">Issue Tracker</a></li>
+      <li><a href="https://github.com/apache/drill" target="_blank">GitHub</a></li>
+    </ul>
+  </li>
+  <li><a href="/faq/">FAQ</a></li>
+  <li><a href="/blog/">Blog</a></li>
+  <li style="width:30px; padding-left: 2px; padding-right:10px"><a href="https://twitter.com/apachedrill" target="_blank"><img src="/images/twitterbw.png" alt="" align="center" width="22" style="padding: 0px 10px 1px 0px;"></a> </li>
+  <li class="l"><span>&nbsp;</span></li>
+  <li class="d"><a href="/download/">Download</a></li>
+</ul>
+</div>
+
+<div class="int_title">
+<h1>Performance</h1>
+
+</div>
+
+<div class="int_text" align="left"><p>Drill is designed from the ground up for high performance on large datasets.
+The following core elements of Drill processing are responsible for Drill&#39;s
+performance:</p>
+
+<p><strong><em>Distributed engine</em></strong></p>
+
+<p>Drill provides a powerful distributed execution engine for processing queries.
+Users can submit requests to any node in the cluster. You can simply add new
+nodes to the cluster to scale for larger volumes of data, support more users
+or to improve performance.</p>
+
+<p><strong><em>Columnar execution</em></strong></p>
+
+<p>Drill optimizes for both columnar storage and execution by using an in-memory
+data model that is hierarchical and columnar. When working with data stored in
+columnar formats such as Parquet, Drill avoids disk access for columns that
+are not involved in an analytic query. Drill also provides an execution layer
+that performs SQL processing directly on columnar data without row
+materialization. The combination of optimizations for columnar storage and
+direct columnar execution significantly lowers memory footprints and provides
+faster execution of BI/Analytic type of workloads.</p>
+
+<p><strong><em>Vectorization</em></strong></p>
+
+<p>Rather than operating on single values from a single table record at one time,
+vectorization in Drill allows the CPU to operate on vectors, referred to as a
+Record Batches. Record Batches are arrays of values from many different
+records. The technical basis for efficiency of vectorized processing is modern
+chip technology with deep-pipelined CPU designs. Keeping all pipelines full to
+achieve efficiency near peak performance is something impossible to achieve in
+traditional database engines, primarily due to code complexity*<em>.</em>*</p>
+
+<p><strong><em>Runtime compilation</em></strong></p>
+
+<p>Runtime compilation is faster compared to the interpreted execution. Drill
+generates highly efficient custom code for every single query for every single
+operator. Here is a quick overview of the Drill compilation/code generation
+process at a glance.</p>
+
+<p><img src="/confluence/download/attachments/40512736/image2014-8-4%2010%3A8%3A58.png?%0Aversion=1&amp;modificationDate=1407172137000&amp;api=v2" alt=""></p>
+
+<p><strong>Optimistic and pipelined query execution</strong></p>
+
+<p>Drill adopts an optimistic execution model to process queries. Drill assumes
+that failures are infrequent within the short span of a query and therefore
+does not spend time creating boundaries or checkpoints to minimize recovery
+time. Failures at node level are handled gracefully. In the instance of a
+single query failure, the query is rerun. Drill execution uses a pipeline
+model where all tasks are scheduled at once. The query execution happens in-
+memory as much as possible to move data through task pipelines, persisting to
+disk only if there is memory overflow.</p>
+</div>
+
+
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2014 The Apache Software Foundation, licensed under the Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Other names appearing on the site may be trademarks of their respective owners.<br/><br/>
+</div>
+</div>
+
+<script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+
+</body>
+</html>



Mime
View raw message