drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [2/3] drill git commit: DRILL-2586
Date Tue, 31 Mar 2015 01:14:51 GMT
http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/design/004-research.md
----------------------------------------------------------------------
diff --git a/_docs/design/004-research.md b/_docs/design/004-research.md
deleted file mode 100644
index 77be828..0000000
--- a/_docs/design/004-research.md
+++ /dev/null
@@ -1,48 +0,0 @@
----
-title: "Useful Research"
-parent: "Design Docs"
----
-## Drill itself
-
-  * Apache Proposal: <http://wiki.apache.org/incubator/DrillProposal>
-  * Mailing List Archive: <http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/>
-  * DrQL ANTLR grammar: <https://gist.github.com/3483314>
-  * Apache Drill, Architecture outlines: <http://www.slideshare.net/jasonfrantz/drill-architecture-20120913>
-
-## Background info
-
-  * Dremel Paper: <http://research.google.com/pubs/pub36632.html>
-  * Dremel Presentation: <http://www.slideshare.net/robertlz/dremel-interactive-analysis-of-webscale-datasets>
-  * Query Language: <http://developers.google.com/bigquery/docs/query-reference>
-  * Protobuf: <http://developers.google.com/protocol-buffers/docs/proto>
-  * Dryad: <http://research.microsoft.com/en-us/projects/dryad/>
-  * SQLServer Query Plan: <http://msdn.microsoft.com/en-us/library/ms191158.aspx>
-  * CStore: <http://db.csail.mit.edu/projects/cstore/>
-  * Vertica (commercial evolution of C-Store): <http://vldb.org/pvldb/vol5/p1790_andrewlamb_vldb2012.pdf>
-  * <http://pdf.aminer.org/000/094/728/database_cracking.pdf>
-  * <http://homepages.cwi.nl/~idreos/NoDBsigmod2012.pdf>
-  * <http://db.csail.mit.edu/projects/cstore/abadiicde2007.pdf>
-  * Hive Architecture: <https://cwiki.apache.org/confluence/display/Hive/Design#Design-HiveArchitecture>
-  * Fast Response in an unreliable world: <http://research.google.com/people/jeff/latency.html>
-  * Column-Oriented Database Systems: <http://www.vldb.org/pvldb/2/vldb09-tutorial6.pdf> (SLIDES: <http://phdopen.mimuw.edu.pl/lato10/boncz_mimuw.pdf>)
-
-## OpenDremel
-
-  * OpenDremel site: <http://code.google.com/p/dremel/>
-  * Design Proposal for Drill: <http://www.slideshare.net/CamuelGilyadov/apache-drill-14071739>
-
-## Dazo (second generation OpenDremel)
-
-  * Dazo repos: <https://github.com/Dazo-org>
-  * ZeroVM (multi-tenant executor): <http://zerovm.org/>
-  * ZeroVM elaboration: <http://news.ycombinator.com/item?id=3746222>
-
-## Rob Grzywinski Dremel adventures
-
-  * <https://github.com/rgrzywinski/field-stripe/>
-
-## Code generation / Physical plan generation
-
-  * <http://www.vldb.org/pvldb/vol4/p539-neumann.pdf> (SLIDES: <http://www.vldb.org/2011/files/slides/research9/rSession9-3.pdf>)
-  * <http://www.vldb.org/pvldb/2/vldb09-327.pdf> (SLIDES: <http://www.slideserve.com/cher/simd-scan-ultra-fast-in-memory-table-scan-using-on-chip-vector-processing-units>)
-

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/design/005-value.md
----------------------------------------------------------------------
diff --git a/_docs/design/005-value.md b/_docs/design/005-value.md
deleted file mode 100644
index 828376a..0000000
--- a/_docs/design/005-value.md
+++ /dev/null
@@ -1,163 +0,0 @@
----
-title: "Value Vectors"
-parent: "Design Docs"
----
-This document defines the data structures required for passing sequences of
-columnar data between [Operators](https://docs.google.com/a/maprtech.com/document/d/1zaxkcrK9mYyfpGwX1kAV80z0PCi8abefL45zOzb97dI/edit#bookmark=id.iip15ful18mm).
-
-## Goals
-
-### Support Operators Written in Multiple Language
-
-ValueVectors should support operators written in C/C++/Assembly. To support
-this, the underlying ByteBuffer will not require modification when passed
-through the JNI interface. The ValueVector will be considered immutable once
-constructed. Endianness has not yet been considered.
-
-### Access
-
-Reading a random element from a ValueVector must be a constant time operation.
-To accomodate, elements are identified by their offset from the start of the
-buffer. Repeated, nullable and variable width ValueVectors utilize in an
-additional fixed width value vector to index each element. Write access is not
-supported once the ValueVector has been constructed by the RecordBatch.
-
-### Efficient Subsets of Value Vectors
-
-When an operator returns a subset of values from a ValueVector, it should
-reuse the original ValueVector. To accomplish this, a level of indirection is
-introduced to skip over certain values in the vector. This level of
-indirection is a sequence of offsets which reference an offset in the original
-ValueVector and the count of subsequent values which are to be included in the
-subset.
-
-### Pooled Allocation
-
-ValueVectors utilize one or more buffers under the covers. These buffers will
-be drawn from a pool. Value vectors are themselves created and destroyed as a
-schema changes during the course of record iteration.
-
-### Homogenous Value Types
-
-Each value in a Value Vector is of the same type. The [Record Batch](https://docs.google.com/a/maprtech.com/document/d/1zaxkcrK9mYyfpGwX1kAV80z0PCi8abefL45zOzb97dI/edit#bookmark=kix.s2xuoqnr8obe) implementation is responsible for
-creating a new Value Vector any time there is a change in schema.
-
-## Definitions
-
-Data Types
-
-The canonical source for value type definitions is the [Drill
-Datatypes](http://bit.ly/15JO9bC) document. The individual types are listed
-under the ‘Basic Data Types’ tab, while the value vector types can be found
-under the ‘Value Vectors’ tab.
-
-Operators
-
-An operator is responsible for transforming a stream of fields. It operates on
-Record Batches or constant values.
-
-Record Batch
-
-A set of field values for some range of records. The batch may be composed of
-Value Vectors, in which case each batch consists of exactly one schema.
-
-Value Vector
-
-The value vector is comprised of one or more contiguous buffers; one which
-stores a sequence of values, and zero or more which store any metadata
-associated with the ValueVector.
-
-## Data Structure
-
-A ValueVector stores values in a ByteBuf, which is a contiguous region of
-memory. Additional levels of indirection are used to support variable value
-widths, nullable values, repeated values and selection vectors. These levels
-of indirection are primarily lookup tables which consist of one or more fixed
-width ValueVectors which may be combined (e.g. for nullable, variable width
-values). A fixed width ValueVector of non-nullable, non-repeatable values does
-not require an indirect lookup; elements can be accessed directly by
-multiplying position by stride.
-
-Fixed Width Values
-
-Fixed width ValueVectors simply contain a packed sequence of values. Random
-access is supported by accessing element n at ByteBuf[0] + Index * Stride,
-where Index is 0-based. The following illustrates the underlying buffer of
-INT4 values [1 .. 6]:
-
-![drill query flow]({{ site.baseurl }}/docs/img/value1.png)
-
-Nullable Values
-
-Nullable values are represented by a vector of bit values. Each bit in the
-vector corresponds to an element in the ValueVector. If the bit is not set,
-the value is NULL. Otherwise the value is retrieved from the underlying
-buffer. The following illustrates a NullableValueVector of INT4 values 2, 3
-and 6:
-
-![drill query flow]({{ site.baseurl }}/docs/img/value2.png)
-  
-### Repeated Values
-
-A repeated ValueVector is used for elements which can contain multiple values
-(e.g. a JSON array). A table of offset and count pairs is used to represent
-each repeated element in the ValueVector. A count of zero means the element
-has no values (note the offset field is unused in this case). The following
-illustrates three fields; one with two values, one with no values, and one
-with a single value:
-
-![drill query flow]({{ site.baseurl }}/docs/img/value3.png)
-
-ValueVector Representation of the equivalent JSON:
-
-x:[1, 2]
-
-x:[ ]
-
-x:[3]
-
-Variable Width Values
-
-Variable width values are stored contiguously in a ByteBuf. Each element is
-represented by an entry in a fixed width ValueVector of offsets. The length of
-an entry is deduced by subtracting the offset of the following field. Because
-of this, the offset table will always contain one more entry than total
-elements, with the last entry pointing to the end of the buffer.
-
-![drill query flow]({{ site.baseurl }}/docs/img/value4.png)  
-
-Repeated Map Vectors
-
-A repeated map vector contains one or more maps (akin to an array of objects
-in JSON). The values of each field in the map are stored contiguously within a
-ByteBuf. To access a specific record, a lookup table of count and offset pairs
-is used. This lookup table points to the first repeated field in each column,
-while the count indicates the maximum number of elements for the column. The
-following example illustrates a RepeatedMap with two records; one with two
-objects, and one with a single object:
-
-![drill query flow]({{ site.baseurl }}/docs/img/value5.png)
-
-ValueVector representation of the equivalent JSON:
-
-x: [ {name:’Sam’, age:1}, {name:’Max’, age:2} ]
-
-x: [ {name:’Joe’, age:3} ]
-
-Selection Vectors
-
-A Selection Vector represents a subset of a ValueVector. It is implemented
-with a list of offsets which identify each element in the ValueVector to be
-included in the SelectionVector. In the case of a fixed width ValueVector, the
-offsets reference the underlying ByteBuf. In the case of a nullable, repeated
-or variable width ValueVector, the offset references the corresponding lookup
-table. The following illustrates a SelectionVector of INT4 (fixed width)
-values 2, 3 and 5 from the original vector of [1 .. 6]:
-
-![drill query flow]({{ site.baseurl }}/docs/img/value6.png)
-
-The following illustrates the same ValueVector with nullable fields:
-
-![drill query flow]({{ site.baseurl }}/docs/img/value7.png)
-
-

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/001-compile.md
----------------------------------------------------------------------
diff --git a/_docs/develop/001-compile.md b/_docs/develop/001-compile.md
deleted file mode 100644
index dea42e9..0000000
--- a/_docs/develop/001-compile.md
+++ /dev/null
@@ -1,37 +0,0 @@
----
-title: "Compiling Drill from Source"
-parent: "Develop Drill"
----
-## Prerequisites
-
-  * Maven 3.0.4 or later
-  * Oracle JDK 7 or later
-
-Run the following commands to verify that you have the correct versions of
-Maven and JDK installed:
-
-    java -version
-    mvn -version
-
-## 1\. Clone the Repository
-
-    git clone https://git-wip-us.apache.org/repos/asf/incubator-drill.git
-
-## 2\. Compile the Code
-
-    cd incubator-drill
-    mvn clean install -DskipTests
-
-## 3\. Explode the Tarball in the Installation Directory
-
-    mkdir ~/compiled-drill
-    tar xvzf distribution/target/*.tar.gz --strip=1 -C ~/compiled-drill
-
-Now that you have Drill installed, you can connect to Drill and query sample
-data or you can connect Drill to your data sources.
-
-  * To connect Drill to your data sources, refer to [Connect to Data Sources](/docs/connect-to-data-sources) for instructions.
-  * To connect to Drill and query sample data, refer to the following topics:
-    * [Start Drill ](/docs/starting-stopping-drill)(For Drill installed in embedded mode)
-    * [Query Data ](/docs/query-data)
-

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/001-develop-drill.md
----------------------------------------------------------------------
diff --git a/_docs/develop/001-develop-drill.md b/_docs/develop/001-develop-drill.md
new file mode 100644
index 0000000..0c6530a
--- /dev/null
+++ b/_docs/develop/001-develop-drill.md
@@ -0,0 +1,9 @@
+---
+title: "Develop Drill"
+parent: "Developer Information"
+---
+To develop Drill, you compile Drill from source code and then set up a project
+in Eclipse for use as your development environment. To review or contribute to
+Drill code, you must complete the steps required to install and use the Drill
+patch review tool.
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/002-contribute.md
----------------------------------------------------------------------
diff --git a/_docs/develop/002-contribute.md b/_docs/develop/002-contribute.md
new file mode 100644
index 0000000..b480b36
--- /dev/null
+++ b/_docs/develop/002-contribute.md
@@ -0,0 +1,10 @@
+---
+title: "Contribute to Drill"
+parent: "Developer Information"
+---
+The Apache Drill community welcomes your support. Please read [Apache Drill
+Contribution Guidelines](/docs/apache-drill-contribution-guidelines) for information about how to contribute to
+the project. If you would like to contribute to the project and need some
+ideas for what to do, please read [Apache Drill Contribution
+Ideas](/docs/apache-drill-contribution-ideas).
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/002-setup.md
----------------------------------------------------------------------
diff --git a/_docs/develop/002-setup.md b/_docs/develop/002-setup.md
deleted file mode 100644
index 19fb554..0000000
--- a/_docs/develop/002-setup.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-title: "Setting Up Your Development Environment"
-parent: "Develop Drill"
----
-TBD
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/003-design.md
----------------------------------------------------------------------
diff --git a/_docs/develop/003-design.md b/_docs/develop/003-design.md
new file mode 100644
index 0000000..e14e3f6
--- /dev/null
+++ b/_docs/develop/003-design.md
@@ -0,0 +1,14 @@
+---
+title: "Design Docs"
+parent: "Developer Information"
+---
+Review the Apache Drill design docs for early descriptions of Apache Drill
+functionality, terms, and goals, and reference the research articles to learn
+about Apache Drill's history:
+
+  * [Drill Plan Syntax](/docs/drill-plan-syntax)
+  * [RPC Overview](/docs/rpc-overview)
+  * [Query Stages](/docs/query-stages)
+  * [Useful Research](/docs/useful-research)
+  * [Value Vectors](/docs/value-vectors)
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/003-patch-tool.md
----------------------------------------------------------------------
diff --git a/_docs/develop/003-patch-tool.md b/_docs/develop/003-patch-tool.md
deleted file mode 100644
index 28c8a54..0000000
--- a/_docs/develop/003-patch-tool.md
+++ /dev/null
@@ -1,160 +0,0 @@
----
-title: "Drill Patch Review Tool"
-parent: "Develop Drill"
----
-  * Drill JIRA and Reviewboard script
-    * 1\. Setup
-    * 2\. Usage
-    * 3\. Upload patch
-    * 4\. Update patch
-  * JIRA command line tool
-    * 1\. Download the JIRA command line package
-    * 2\. Configure JIRA username and password
-  * Reviewboard
-    * 1\. Install the post-review tool
-    * 2\. Configure Stuff
-  * FAQ
-    * When I run the script, it throws the following error and exits
-    * When I run the script, it throws the following error and exits
-
-### Drill JIRA and Reviewboard script
-
-#### 1\. Setup
-
-  1. Follow instructions [here](/docs/drill-patch-review-tool#jira-command-line-tool) to setup the jira-python package
-  2. Follow instructions [here](/docs/drill-patch-review-tool#reviewboard) to setup the reviewboard python tools
-  3. Install the argparse module 
-  
-        On Linux -> sudo yum install python-argparse
-        On Mac -> sudo easy_install argparse
-
-#### 2\. Usage
-
-	nnarkhed-mn: nnarkhed$ python drill-patch-review.py --help
-	usage: drill-patch-review.py [-h] -b BRANCH -j JIRA [-s SUMMARY]
-	                             [-d DESCRIPTION] [-r REVIEWBOARD] [-t TESTING]
-	                             [-v VERSION] [-db] -rbu REVIEWBOARDUSER -rbp REVIEWBOARDPASSWORD
-	 
-	Drill patch review tool
-	 
-	optional arguments:
-	  -h, --help            show this help message and exit
-	  -b BRANCH, --branch BRANCH
-	                        Tracking branch to create diff against
-	  -j JIRA, --jira JIRA  JIRA corresponding to the reviewboard
-	  -s SUMMARY, --summary SUMMARY
-	                        Summary for the reviewboard
-	  -d DESCRIPTION, --description DESCRIPTION
-	                        Description for reviewboard
-	  -r REVIEWBOARD, --rb REVIEWBOARD
-	                        Review board that needs to be updated
-	  -t TESTING, --testing-done TESTING
-	                        Text for the Testing Done section of the reviewboard
-	  -v VERSION, --version VERSION
-	                        Version of the patch
-	  -db, --debug          Enable debug mode
-	  -rbu, --reviewboard-user Reviewboard user name
-	  -rbp, --reviewboard-password Reviewboard password
-
-#### 3\. Upload patch
-
-  1. Specify the branch against which the patch should be created (-b)
-  2. Specify the corresponding JIRA (-j)
-  3. Specify an **optional** summary (-s) and description (-d) for the reviewboard
-
-Example:
-
-    python drill-patch-review.py -b origin/master -j DRILL-241 -rbu tnachen -rbp password
-
-#### 4\. Update patch
-
-  1. Specify the branch against which the patch should be created (-b)
-  2. Specify the corresponding JIRA (--jira)
-  3. Specify the rb to be updated (-r)
-  4. Specify an **optional** summary (-s) and description (-d) for the reviewboard, if you want to update it
-  5. Specify an **optional** version of the patch. This will be appended to the jira to create a file named JIRA-<version>.patch. The purpose is to be able to upload multiple patches to the JIRA. This has no bearing on the reviewboard update.
-
-Example:
-
-    python drill-patch-review.py -b origin/master -j DRILL-241 -r 14081 rbp tnachen -rbp password
-
-### JIRA command line tool
-
-#### 1\. Download the JIRA command line package
-
-Install the jira-python package.
-
-    sudo easy_install jira-python
-
-#### 2\. Configure JIRA username and password
-
-Include a jira.ini file in your $HOME directory that contains your Apache JIRA
-username and password.
-
-	nnarkhed-mn:~ nnarkhed$ cat ~/jira.ini
-	user=nehanarkhede
-	password=***********
-
-### Reviewboard
-
-This is a quick tutorial on using [Review Board](https://reviews.apache.org)
-with Drill.
-
-#### 1\. Install the post-review tool
-
-If you are on RHEL, Fedora or CentOS, follow these steps:
-
-	sudo yum install python-setuptools
-	sudo easy_install -U RBTools
-
-If you are on Mac, follow these steps:
-
-	sudo easy_install -U setuptools
-	sudo easy_install -U RBTools
-
-For other platforms, follow the [instructions](http://www.reviewboard.org/docs/manual/dev/users/tools/post-review/) to
-setup the post-review tool.
-
-#### 2\. Configure Stuff
-
-Then you need to configure a few things to make it work.
-
-First set the review board url to use. You can do this from in git:
-
-    git config reviewboard.url https://reviews.apache.org
-
-If you checked out using the git wip http url that confusingly won't work with
-review board. So you need to configure an override to use the non-http url.
-You can do this by adding a config file like this:
-
-	jkreps$ cat ~/.reviewboardrc
-	REPOSITORY = 'git://git.apache.org/incubator-drill.git'
-	TARGET_GROUPS = 'drill-git'
-GUESS_FIELDS = True
-
-
-
-### FAQ
-
-#### When I run the script, it throws the following error and exits
-
-    nnarkhed$python drill-patch-review.py -b trunk -j DRILL-241
-    There don't seem to be any diffs
-
-There are two reasons for this:
-
-  * The code is not checked into your local branch
-  * The -b branch is not pointing to the remote branch. In the example above, "trunk" is specified as the branch, which is the local branch. The correct value for the -b (--branch) option is the remote branch. "git branch -r" gives the list of the remote branch names.
-
-#### When I run the script, it throws the following error and exits
-
-Error uploading diff
- 
-Your review request still exists, but the diff is not attached.
-
-One of the most common root causes of this error are that the git remote
-branches are not up-to-date. Since the script already does that, it is
-probably due to some other problem. You can run the script with the --debug
-option that will make post-review run in the debug mode and list the root
-cause of the issue.
-

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/contribute/001-guidelines.md
----------------------------------------------------------------------
diff --git a/_docs/develop/contribute/001-guidelines.md b/_docs/develop/contribute/001-guidelines.md
new file mode 100644
index 0000000..7361e5c
--- /dev/null
+++ b/_docs/develop/contribute/001-guidelines.md
@@ -0,0 +1,228 @@
+---
+title: "Apache Drill Contribution Guidelines"
+parent: "Contribute to Drill"
+---
+## How to Contribute to Apache Drill
+
+Disclaimer: These contribution guidelines are largely based on Apache Hive
+contribution guidelines.
+
+This page describes the mechanics of _how_ to contribute software to Apache
+Drill. For ideas about _what_ you might contribute, please see open tickets in
+[Jira](https://issues.apache.org/jira/browse/DRILL).
+
+These guidelines include the following topics:
+
+* Getting the source code
+  * Making Changes
+    * Coding Convention
+    * Formatter configuration
+    * Understanding Maven
+    * Creating a patch
+    * Applying a patch
+  * Where is a good place to start contributing?
+  * Contributing your work
+  * JIRA Guidelines
+  * See Also
+
+### Getting the source code
+
+First, you need the Drill source code.
+
+Get the source code on your local drive using [Git](https://git-wip-
+us.apache.org/repos/asf/incubator-drill.git). Most development is done on
+"master":
+
+    git clone https://git-wip-us.apache.org/repos/asf/drill.git
+
+### Making Changes
+
+Before you start, send a message to the [Drill developer mailing list](http
+://mail-archives.apache.org/mod_mbox/incubator-drill-dev/), or file a bug
+report in [JIRA](https://issues.apache.org/jira/browse/DRILL). Describe your
+proposed changes and check that they fit in with what others are doing and
+have planned for the project. Be patient, it may take folks a while to
+understand your requirements.
+
+Modify the source code and add some features using your favorite IDE.
+
+#### Coding Convention
+
+Please take care about the following points
+
+  * All public classes and methods should have informative [Javadoc comments](http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html).
+    * Do not use @author tags.
+  * Code should be formatted according to [Sun's conventions](http://www.oracle.com/technetwork/java/codeconv-138413.html), with one exception:
+    * Indent two (2) spaces per level, not four (4).
+    * Line length limit is 120 chars, instead of 80 chars.
+  * Contributions should not introduce new Checkstyle violations.
+  * Contributions should pass existing unit tests.
+  * New unit tests should be provided to demonstrate bugs and fixes. [JUnit](http://www.junit.org) 4.1 is our test framework:
+    * You must implement a class that contain test methods annotated with JUnit's 4.x @Test annotation and whose class name ends with `Test`.
+    * Define methods within your class whose names begin with `test`, and call JUnit's many assert methods to verify conditions; these methods will be executed when you run `mvn clean test`.
+
+#### Formatter configuration
+
+Setting up IDE formatters is recommended and can be done by importing the
+following settings into your browser:
+
+IntelliJ IDEA formatter: [settings
+jar](https://cwiki.apache.org/confluence/download/attachments/30757399/idea-settings.jar?version=1&modificationDate=1363022308000&api=v2)
+
+Eclipse: [formatter xml from HBase](https://issues.apache.org/jira/secure/atta
+chment/12474245/eclipse_formatter_apache.xml)
+
+#### Understanding Maven
+
+Drill is built by Maven, a Java build tool.
+
+  * Good Maven tutorial: <http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html>
+
+To build Drill, run
+     
+    mvn clean install 
+    
+
+#### Creating a patch
+
+Check to see what files you have modified:
+
+    git status
+
+Add any new files with:
+
+    git add .../MyNewClass.java
+	git add .../TestMyNewClass.java
+	git add .../XXXXXX.q
+	git add .../XXXXXX.q.out
+
+In order to create a patch, type (from the base directory of drill):
+
+    git format-patch origin/master --stdout > DRILL-1234.1.patch.txt
+
+This will report all modifications done on Drill sources on your local disk
+and save them into the _DRILL-1234.1.patch.txt_ file. Read the patch file.
+Make sure it includes ONLY the modifications required to fix a single issue.
+
+Please do not:
+
+  * reformat code unrelated to the bug being fixed: formatting changes should be separate patches/commits.
+  * comment out code that is now obsolete: just remove it.
+  * insert comments around each change, marking the change: folks can use subversion to figure out what's changed and by whom.
+  * make things public which are not required by end users.
+
+Please do:
+
+  * try to adhere to the coding style of files you edit;
+  * comment code whose function or rationale is not obvious;
+  * update documentation (e.g., _package.html_ files, this wiki, etc.)
+
+Updating a patch
+
+For patch updates, our convention is to number them like
+DRILL-1856.1.patch.txt, DRILL-1856.2.patch.txt, etc. And then click the
+"Submit Patch" button again when a new one is uploaded; this makes sure it
+gets back into the review queue. Appending '.txt' to the patch file name makes
+it easy to quickly view the contents of the patch in a web browser.
+
+#### Applying a patch
+
+To apply a patch either you generated or found from JIRA, you can issue
+
+    git am < cool_patch.patch
+
+if you just want to check whether the patch applies you can run patch with
+--dry-run option.
+
+  
+
+####Review Process
+
+  * Use Hadoop's [code review checklist](http://wiki.apache.org/hadoop/CodeReviewChecklist) as a rough guide when doing reviews.
+  * In JIRA, use attach file to notify that you've submitted a patch for that issue.
+  * Create a Review Request in [Review Board](https://reviews.apache.org/r/). The review request's name should start with the JIRA issue number (e.g. DRILL-XX) and should be assigned to the "drill-git" group.
+  * If a committer requests changes, set the issue status to 'Resume Progress', then once you're ready, submit an updated patch with necessary fixes and then request another round of review with 'Submit Patch' again.
+  * Once your patch is accepted, be sure to upload a final version which grants rights to the ASF.
+
+## Where is a good place to start contributing?
+
+After getting the source code, building and running a few simple queries, one
+of the simplest places to start is to implement a DrillFunc.  
+DrillFuncs is way that Drill express all scalar functions (UDF or system).  
+First you can put together a JIRA for one of the DrillFunc's we don't yet have
+but should (referencing the capabilities of something like Postgres  
+or SQL Server). Then try to implement one.
+
+One example DrillFunc:
+
+[https://github.com/apache/incubator-
+drill/blob/103072a619741d5e228fdb181501ec2f82e111a3/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ComparisonFunction
+s.java](https://github.com/apache/incubator-
+drill/blob/103072a619741d5e228fdb181501ec2f82e111a3/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ComparisonFunction
+s.java)
+
+Also one can visit the JIRA issues and implement one of those too. A list of
+functions which need to be implemented can be found
+[here](https://docs.google.com/spreadsheet/ccc?key=0AgAGbQ6asvQ-
+dDRrUUxVSVlMVXRtanRoWk9JcHgteUE&usp=sharing#gid=0) (WIP).
+
+More contribution ideas are located on the [Contribution Ideas](/docs/apache-drill-contribution-ideas) page.
+
+### Contributing your work
+
+Finally, patches should be _attached_ to an issue report in
+[JIRA](http://issues.apache.org/jira/browse/DRILL) via the **Attach File**
+link on the issue's JIRA. Please add a comment that asks for a code review.
+Please note that the attachment should be granted license to ASF for inclusion
+in ASF works (as per the [Apache
+License](http://www.apache.org/licenses/LICENSE-2.0)).
+
+Folks should run `mvn clean install` before submitting a patch. Tests should
+all pass. If your patch involves performance optimizations, they should be
+validated by benchmarks that demonstrate an improvement.
+
+If your patch creates an incompatibility with the latest major release, then
+you must set the **Incompatible change** flag on the issue's JIRA 'and' fill
+in the **Release Note** field with an explanation of the impact of the
+incompatibility and the necessary steps users must take.
+
+If your patch implements a major feature or improvement, then you must fill in
+the **Release Note** field on the issue's JIRA with an explanation of the
+feature that will be comprehensible by the end user.
+
+A committer should evaluate the patch within a few days and either: commit it;
+or reject it with an explanation.
+
+Please be patient. Committers are busy people too. If no one responds to your
+patch after a few days, please make friendly reminders. Please incorporate
+other's suggestions into your patch if you think they're reasonable. Finally,
+remember that even a patch that is not committed is useful to the community.
+
+Should your patch receive a "-1" select the **Resume Progress** on the issue's
+JIRA, upload a new patch with necessary fixes, and then select the **Submit
+Patch** link again.
+
+Committers: for non-trivial changes, it is best to get another committer to
+review your patches before commit. Use **Submit Patch** link like other
+contributors, and then wait for a "+1" from another committer before
+committing. Please also try to frequently review things in the patch queue.
+
+### JIRA Guidelines
+
+Please comment on issues in JIRA, making their concerns known. Please also
+vote for issues that are a high priority for you.
+
+Please refrain from editing descriptions and comments if possible, as edits
+spam the mailing list and clutter JIRA's "All" display, which is otherwise
+very useful. Instead, preview descriptions and comments using the preview
+button (on the right) before posting them. Keep descriptions brief and save
+more elaborate proposals for comments, since descriptions are included in
+JIRA's automatically sent messages. If you change your mind, note this in a
+new comment, rather than editing an older comment. The issue should preserve
+this history of the discussion.
+
+### See Also
+
+  * [Apache contributor documentation](http://www.apache.org/dev/contributors.html)
+  * [Apache voting documentation](http://www.apache.org/foundation/voting.html)
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/contribute/002-ideas.md
----------------------------------------------------------------------
diff --git a/_docs/develop/contribute/002-ideas.md b/_docs/develop/contribute/002-ideas.md
new file mode 100644
index 0000000..2270112
--- /dev/null
+++ b/_docs/develop/contribute/002-ideas.md
@@ -0,0 +1,158 @@
+---
+title: "Apache Drill Contribution Ideas"
+parent: "Contribute to Drill"
+---
+  * Fixing JIRAs
+  * SQL functions 
+  * Support for new file format readers/writers
+  * Support for new data sources
+  * New query language parsers
+  * Application interfaces
+    * BI Tool testing
+  * General CLI improvements 
+  * Eco system integrations
+    * MapReduce
+    * Hive views
+    * YARN
+    * Spark
+    * Hue
+    * Phoenix
+
+## Fixing JIRAs
+
+This is a good place to begin if you are new to Drill. Feel free to pick
+issues from the Drill JIRA list. When you pick an issue, assign it to
+yourself, inform the team, and start fixing it.
+
+For any questions, seek help from the team by sending email to [drill-
+dev@incubator.apache.org](mailto:drill-dev@incubator.apache.org).
+
+[https://issues.apache.org/jira/browse/DRILL/?selectedTab=com.atlassian.jira
+.jira-projects-plugin:summary-panel](https://issues.apache.org/jira/browse/DRILL/?selectedTab=com.atlassian.jira
+.jira-projects-plugin:summary-panel)
+
+## SQL functions
+
+One of the next simple places to start is to implement a DrillFunc.
DrillFuncs
+is way that Drill express all scalar functions (UDF or system).
 First you can
+put together a JIRA for one of the DrillFunc's we don't yet have but should
+(referencing the capabilities of something like Postgres
or SQL Server or your
+own use case). Then try to implement one.
+
+One example DrillFunc:  
+[https://github.com/apache/incubator-
+drill/blob/103072a619741d5e228fdb181501ec2f82e111a3/sandbox/prototype/exec
+/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ComparisonFunction
+s.java](https://github.com/apache/incubator-
+drill/blob/103072a619741d5e228fdb181501ec2f82e111a3/sandbox/prototype/exec
+/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ComparisonFunction
+s.java)** **
+
+**Additional ideas on functions that can be added to SQL support**
+
+  * Madlib integration
+  * Machine learning functions
+  * Approximate aggregate functions (such as what is available in BlinkDB)
+
+## Support for new file format readers/writers
+
+Currently Drill supports text, JSON and Parquet file formats natively when
+interacting with file system. More readers/writers can be introduced by
+implementing custom storage plugins. Example formats include below.
+
+  * AVRO
+  * Sequence
+  * RC
+  * ORC
+  * Protobuf
+  * XML
+  * Thrift
+  * ....
+
+## Support for new data sources
+
+Implement custom storage plugins for the following non-Hadoop data sources:
+
+  * NoSQL databases (such as Mongo, Cassandra, Couch etc)
+  * Search engines (such as Solr, Lucidworks, Elastic Search etc)
+  * SQL databases (MySQL< PostGres etc)
+  * Generic JDBC/ODBC data sources
+  * HTTP URL
+  * \----
+
+## New query language parsers
+
+Drill exposes strongly typed JSON APIs for logical and physical plans (plan
+syntax at [https://docs.google.com/a/maprtech.com/document/d/1QTL8warUYS2KjldQ
+rGUse7zp8eA72VKtLOHwfXy6c7I/edit#heading=h.n9gdb1ek71hf](https://docs.google.com/a/maprtech.com/document/d/1QTL8warUYS2KjldQ
+rGUse7zp8eA72VKtLOHwfXy6c7I/edit#heading=h.n9gdb1ek71hf) ). Drill provides a
+SQL language parser today, but any language parser that can generate
+logical/physical plans can use Drill's power on the backend as the distributed
+low latency query execution engine along with its support for self-describing
+data and complex/multi-structured data.
+
+  * Pig parser : Use Pig as the language to query data from Drill. Great for existing Pig users.
+  * Hive parser : Use HiveQL as the language to query data from Drill. Great for existing Hive users.
+
+## Application interfaces
+
+Drill currently provides JDBC/ODBC drivers for the applications to interact
+along with a basic version of REST API and a C++ API. The following list
+provides a few possible application interface opportunities:
+
+  * Enhancements to REST APIs (<https://issues.apache.org/jira/browse/DRILL-77>)
+  * Expose Drill tables/views as REST APIs
+  * Language drivers for Drill (python etc)
+  * Thrift support
+  * ....
+
+### BI Tool testing
+
+Drill provides JDBC/ODBC drivers to connect to BI tools. We need to make sure
+Drill works with all major BI tools. Doing a quick sanity testing with your
+favorite BI tool is a good place to learn Drill and also uncover issues in
+being able to do so.
+
+## General CLI improvements
+
+Currently Drill uses SQLLine as the CLI. The goal of this effort is to improve
+the CLI experience by adding functionality such as execute statements from a
+file, output results to a file, display version information, and so on.
+
+## Eco system integrations
+
+### MapReduce
+
+Allow using result set from Drill queries as input to the Hadoop/MapReduce
+jobs.
+
+### Hive views
+
+Query data from existing Hive views using Drill queries. Drill needs to parse
+the HiveQL and translate them appropriately (into Drill's SQL or
+logical/physical plans) to execute the requests.
+
+### YARN
+
+[https://issues.apache.org/jira/browse/_DRILL_-1170](https://issues.apache.org
+/jira/browse/DRILL-1170)
+
+## Spark
+
+Provide ability to invoke Drill queries as part of Apache Spark programs. This
+gives ability for Spark developers/users to leverage Drill richness of the
+query layer , for data source access and as low latency execution engine.
+
+### Hue
+
+Hue is a GUI for users to interact with various Hadoop eco system components
+(such as Hive, Oozie, Pig, HBase, Impala ...). The goal of this project is to
+expose Drill as an application inside Hue so users can explore Drill metadata
+and do SQL queries.
+
+### Phoenix
+
+Phoenix provides a low latency query layer on HBase for operational
+applications. The goal of this effort is to explore opportunities for
+integrating Phoenix with Drill.
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/design/001-plan.md
----------------------------------------------------------------------
diff --git a/_docs/develop/design/001-plan.md b/_docs/develop/design/001-plan.md
new file mode 100644
index 0000000..67e2290
--- /dev/null
+++ b/_docs/develop/design/001-plan.md
@@ -0,0 +1,25 @@
+---
+title: "Drill Plan Syntax"
+parent: "Design Docs"
+---
+### Whats the plan?
+
+This section is about the end-to-end plan flow for Drill. The incoming query
+to Drill can be a SQL 2003 query/DrQL or MongoQL. The query is converted to a
+_Logical Plan_ that is a Drill's internal representation of the query
+(language-agnostic). Drill then uses its optimization rules over the Logical
+Plan to optimize it for best performance and crafts out a _Physical Plan_. The
+Physical Plan is the actual plan the Drill then executes for the final data
+processing. Below is a diagram to illustrate the flow:
+
+![drill query flow]({{ site.baseurl }}/docs/img/slide-15-638.png)
+
+**The Logical Plan** describes the abstract data flow of a language independent query i.e. it would be a representation of the input query which would not be dependent on the actual input query language. It generally tries to work with primitive operations without focus on optimization. This makes it more verbose than traditional query languages. This is to allow a substantial level of flexibility in defining higher-level query language features. It would be forwarded to the optimizer to get a physical plan.
+
+**The Physical Plan** is often called the execution plan, since it is the input to the execution engine. Its a description of the physical operations the execution engine will undertake to get the desired result. It is the output of the query planner and is a transformation of the logical plan after applying the optimization rules.
+
+Typically, the physical and execution plans will be represented using the same
+JSON format as the logical plan.
+
+**Detailed document**: Here is a document that explains the Drill logical & physical plans in full detail. [Drill detailed plan syntax document](https://docs.google.com/document/d/1QTL8warUYS2KjldQrGUse7zp8eA72VKtLOHwfXy6c7I/edit).
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/design/002-rpc.md
----------------------------------------------------------------------
diff --git a/_docs/develop/design/002-rpc.md b/_docs/develop/design/002-rpc.md
new file mode 100644
index 0000000..05cb1d6
--- /dev/null
+++ b/_docs/develop/design/002-rpc.md
@@ -0,0 +1,19 @@
+---
+title: "RPC Overview"
+parent: "Design Docs"
+---
+Drill leverages the Netty 4 project as an RPC underlayment. From there, we
+built a simple protobuf based communication layer optimized to minimize the
+requirement for on heap data transformations. Both client and server utilize
+the CompleteRpcMessage protobuf envelope to communicate requests, responses
+and errors. The communication model is that each endpoint sends a stream of
+CompleteRpcMessages to its peer. The CompleteRpcMessage is prefixed by a
+protobuf encoded length.
+
+CompleteRpcMessage is broken into three key components: RpcHeader, Protobuf
+Body (bytes), RawBody (bytes).
+
+RpcHeader has the following fields:
+
+Drillbits communicate through the BitCom intermediary. BitCom manages...
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/design/003-query-stages.md
----------------------------------------------------------------------
diff --git a/_docs/develop/design/003-query-stages.md b/_docs/develop/design/003-query-stages.md
new file mode 100644
index 0000000..5c54249
--- /dev/null
+++ b/_docs/develop/design/003-query-stages.md
@@ -0,0 +1,42 @@
+---
+title: "Query Stages"
+parent: "Design Docs"
+---
+## Overview
+
+Apache Drill is a system for interactive analysis of large-scale datasets. It
+was designed to allow users to query across multiple large big data systems
+using traditional query technologies such as SQL. It is built as a flexible
+framework to support a wide variety of data operations, query languages and
+storage engines.
+
+## Query Parsing
+
+A Drillbit is capable of parsing a provided query into a logical plan. In
+theory, Drill is capable of parsing a large range of query languages. At
+launch, this will likely be restricted to an enhanced SQL2003 language.
+
+## Physical Planning
+
+Once a query is parsed into a logical plan, a Drillbit will then translate the
+plan into a physical plan. The physical plan will then be optimized for
+performance. Since plan optimization can be computationally intensive, a
+distributed in-memory cache will provide LRU retrieval of previously generated
+optimized plans to speed query execution.
+
+## Execution Planning
+
+Once a physical plan is generated, the physical plan is then rendered into a
+set of detailed executional plan fragments (EPFs). This rendering is based on
+available resources, cluster load, query priority and detailed information
+about data distribution. In the case of large clusters, a subset of nodes will
+be responsible for rendering the EPFs. Shared state will be managed through
+the use of a distributed in-memory cache.
+
+## Execution Operation
+
+Query execution starts with each Drillbit being provided with one or more EPFs
+associated with query execution. A portion of these EPFs may be identified as
+initial EPFs and thus they are executed immediately. Other EPFs are executed
+as data flows into them.
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/design/004-research.md
----------------------------------------------------------------------
diff --git a/_docs/develop/design/004-research.md b/_docs/develop/design/004-research.md
new file mode 100644
index 0000000..77be828
--- /dev/null
+++ b/_docs/develop/design/004-research.md
@@ -0,0 +1,48 @@
+---
+title: "Useful Research"
+parent: "Design Docs"
+---
+## Drill itself
+
+  * Apache Proposal: <http://wiki.apache.org/incubator/DrillProposal>
+  * Mailing List Archive: <http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/>
+  * DrQL ANTLR grammar: <https://gist.github.com/3483314>
+  * Apache Drill, Architecture outlines: <http://www.slideshare.net/jasonfrantz/drill-architecture-20120913>
+
+## Background info
+
+  * Dremel Paper: <http://research.google.com/pubs/pub36632.html>
+  * Dremel Presentation: <http://www.slideshare.net/robertlz/dremel-interactive-analysis-of-webscale-datasets>
+  * Query Language: <http://developers.google.com/bigquery/docs/query-reference>
+  * Protobuf: <http://developers.google.com/protocol-buffers/docs/proto>
+  * Dryad: <http://research.microsoft.com/en-us/projects/dryad/>
+  * SQLServer Query Plan: <http://msdn.microsoft.com/en-us/library/ms191158.aspx>
+  * CStore: <http://db.csail.mit.edu/projects/cstore/>
+  * Vertica (commercial evolution of C-Store): <http://vldb.org/pvldb/vol5/p1790_andrewlamb_vldb2012.pdf>
+  * <http://pdf.aminer.org/000/094/728/database_cracking.pdf>
+  * <http://homepages.cwi.nl/~idreos/NoDBsigmod2012.pdf>
+  * <http://db.csail.mit.edu/projects/cstore/abadiicde2007.pdf>
+  * Hive Architecture: <https://cwiki.apache.org/confluence/display/Hive/Design#Design-HiveArchitecture>
+  * Fast Response in an unreliable world: <http://research.google.com/people/jeff/latency.html>
+  * Column-Oriented Database Systems: <http://www.vldb.org/pvldb/2/vldb09-tutorial6.pdf> (SLIDES: <http://phdopen.mimuw.edu.pl/lato10/boncz_mimuw.pdf>)
+
+## OpenDremel
+
+  * OpenDremel site: <http://code.google.com/p/dremel/>
+  * Design Proposal for Drill: <http://www.slideshare.net/CamuelGilyadov/apache-drill-14071739>
+
+## Dazo (second generation OpenDremel)
+
+  * Dazo repos: <https://github.com/Dazo-org>
+  * ZeroVM (multi-tenant executor): <http://zerovm.org/>
+  * ZeroVM elaboration: <http://news.ycombinator.com/item?id=3746222>
+
+## Rob Grzywinski Dremel adventures
+
+  * <https://github.com/rgrzywinski/field-stripe/>
+
+## Code generation / Physical plan generation
+
+  * <http://www.vldb.org/pvldb/vol4/p539-neumann.pdf> (SLIDES: <http://www.vldb.org/2011/files/slides/research9/rSession9-3.pdf>)
+  * <http://www.vldb.org/pvldb/2/vldb09-327.pdf> (SLIDES: <http://www.slideserve.com/cher/simd-scan-ultra-fast-in-memory-table-scan-using-on-chip-vector-processing-units>)
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/design/005-value.md
----------------------------------------------------------------------
diff --git a/_docs/develop/design/005-value.md b/_docs/develop/design/005-value.md
new file mode 100644
index 0000000..828376a
--- /dev/null
+++ b/_docs/develop/design/005-value.md
@@ -0,0 +1,163 @@
+---
+title: "Value Vectors"
+parent: "Design Docs"
+---
+This document defines the data structures required for passing sequences of
+columnar data between [Operators](https://docs.google.com/a/maprtech.com/document/d/1zaxkcrK9mYyfpGwX1kAV80z0PCi8abefL45zOzb97dI/edit#bookmark=id.iip15ful18mm).
+
+## Goals
+
+### Support Operators Written in Multiple Language
+
+ValueVectors should support operators written in C/C++/Assembly. To support
+this, the underlying ByteBuffer will not require modification when passed
+through the JNI interface. The ValueVector will be considered immutable once
+constructed. Endianness has not yet been considered.
+
+### Access
+
+Reading a random element from a ValueVector must be a constant time operation.
+To accomodate, elements are identified by their offset from the start of the
+buffer. Repeated, nullable and variable width ValueVectors utilize in an
+additional fixed width value vector to index each element. Write access is not
+supported once the ValueVector has been constructed by the RecordBatch.
+
+### Efficient Subsets of Value Vectors
+
+When an operator returns a subset of values from a ValueVector, it should
+reuse the original ValueVector. To accomplish this, a level of indirection is
+introduced to skip over certain values in the vector. This level of
+indirection is a sequence of offsets which reference an offset in the original
+ValueVector and the count of subsequent values which are to be included in the
+subset.
+
+### Pooled Allocation
+
+ValueVectors utilize one or more buffers under the covers. These buffers will
+be drawn from a pool. Value vectors are themselves created and destroyed as a
+schema changes during the course of record iteration.
+
+### Homogenous Value Types
+
+Each value in a Value Vector is of the same type. The [Record Batch](https://docs.google.com/a/maprtech.com/document/d/1zaxkcrK9mYyfpGwX1kAV80z0PCi8abefL45zOzb97dI/edit#bookmark=kix.s2xuoqnr8obe) implementation is responsible for
+creating a new Value Vector any time there is a change in schema.
+
+## Definitions
+
+Data Types
+
+The canonical source for value type definitions is the [Drill
+Datatypes](http://bit.ly/15JO9bC) document. The individual types are listed
+under the ‘Basic Data Types’ tab, while the value vector types can be found
+under the ‘Value Vectors’ tab.
+
+Operators
+
+An operator is responsible for transforming a stream of fields. It operates on
+Record Batches or constant values.
+
+Record Batch
+
+A set of field values for some range of records. The batch may be composed of
+Value Vectors, in which case each batch consists of exactly one schema.
+
+Value Vector
+
+The value vector is comprised of one or more contiguous buffers; one which
+stores a sequence of values, and zero or more which store any metadata
+associated with the ValueVector.
+
+## Data Structure
+
+A ValueVector stores values in a ByteBuf, which is a contiguous region of
+memory. Additional levels of indirection are used to support variable value
+widths, nullable values, repeated values and selection vectors. These levels
+of indirection are primarily lookup tables which consist of one or more fixed
+width ValueVectors which may be combined (e.g. for nullable, variable width
+values). A fixed width ValueVector of non-nullable, non-repeatable values does
+not require an indirect lookup; elements can be accessed directly by
+multiplying position by stride.
+
+Fixed Width Values
+
+Fixed width ValueVectors simply contain a packed sequence of values. Random
+access is supported by accessing element n at ByteBuf[0] + Index * Stride,
+where Index is 0-based. The following illustrates the underlying buffer of
+INT4 values [1 .. 6]:
+
+![drill query flow]({{ site.baseurl }}/docs/img/value1.png)
+
+Nullable Values
+
+Nullable values are represented by a vector of bit values. Each bit in the
+vector corresponds to an element in the ValueVector. If the bit is not set,
+the value is NULL. Otherwise the value is retrieved from the underlying
+buffer. The following illustrates a NullableValueVector of INT4 values 2, 3
+and 6:
+
+![drill query flow]({{ site.baseurl }}/docs/img/value2.png)
+  
+### Repeated Values
+
+A repeated ValueVector is used for elements which can contain multiple values
+(e.g. a JSON array). A table of offset and count pairs is used to represent
+each repeated element in the ValueVector. A count of zero means the element
+has no values (note the offset field is unused in this case). The following
+illustrates three fields; one with two values, one with no values, and one
+with a single value:
+
+![drill query flow]({{ site.baseurl }}/docs/img/value3.png)
+
+ValueVector Representation of the equivalent JSON:
+
+x:[1, 2]
+
+x:[ ]
+
+x:[3]
+
+Variable Width Values
+
+Variable width values are stored contiguously in a ByteBuf. Each element is
+represented by an entry in a fixed width ValueVector of offsets. The length of
+an entry is deduced by subtracting the offset of the following field. Because
+of this, the offset table will always contain one more entry than total
+elements, with the last entry pointing to the end of the buffer.
+
+![drill query flow]({{ site.baseurl }}/docs/img/value4.png)  
+
+Repeated Map Vectors
+
+A repeated map vector contains one or more maps (akin to an array of objects
+in JSON). The values of each field in the map are stored contiguously within a
+ByteBuf. To access a specific record, a lookup table of count and offset pairs
+is used. This lookup table points to the first repeated field in each column,
+while the count indicates the maximum number of elements for the column. The
+following example illustrates a RepeatedMap with two records; one with two
+objects, and one with a single object:
+
+![drill query flow]({{ site.baseurl }}/docs/img/value5.png)
+
+ValueVector representation of the equivalent JSON:
+
+x: [ {name:’Sam’, age:1}, {name:’Max’, age:2} ]
+
+x: [ {name:’Joe’, age:3} ]
+
+Selection Vectors
+
+A Selection Vector represents a subset of a ValueVector. It is implemented
+with a list of offsets which identify each element in the ValueVector to be
+included in the SelectionVector. In the case of a fixed width ValueVector, the
+offsets reference the underlying ByteBuf. In the case of a nullable, repeated
+or variable width ValueVector, the offset references the corresponding lookup
+table. The following illustrates a SelectionVector of INT4 (fixed width)
+values 2, 3 and 5 from the original vector of [1 .. 6]:
+
+![drill query flow]({{ site.baseurl }}/docs/img/value6.png)
+
+The following illustrates the same ValueVector with nullable fields:
+
+![drill query flow]({{ site.baseurl }}/docs/img/value7.png)
+
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/develop-drill/001-compile.md
----------------------------------------------------------------------
diff --git a/_docs/develop/develop-drill/001-compile.md b/_docs/develop/develop-drill/001-compile.md
new file mode 100644
index 0000000..dea42e9
--- /dev/null
+++ b/_docs/develop/develop-drill/001-compile.md
@@ -0,0 +1,37 @@
+---
+title: "Compiling Drill from Source"
+parent: "Develop Drill"
+---
+## Prerequisites
+
+  * Maven 3.0.4 or later
+  * Oracle JDK 7 or later
+
+Run the following commands to verify that you have the correct versions of
+Maven and JDK installed:
+
+    java -version
+    mvn -version
+
+## 1\. Clone the Repository
+
+    git clone https://git-wip-us.apache.org/repos/asf/incubator-drill.git
+
+## 2\. Compile the Code
+
+    cd incubator-drill
+    mvn clean install -DskipTests
+
+## 3\. Explode the Tarball in the Installation Directory
+
+    mkdir ~/compiled-drill
+    tar xvzf distribution/target/*.tar.gz --strip=1 -C ~/compiled-drill
+
+Now that you have Drill installed, you can connect to Drill and query sample
+data or you can connect Drill to your data sources.
+
+  * To connect Drill to your data sources, refer to [Connect to Data Sources](/docs/connect-to-data-sources) for instructions.
+  * To connect to Drill and query sample data, refer to the following topics:
+    * [Start Drill ](/docs/starting-stopping-drill)(For Drill installed in embedded mode)
+    * [Query Data ](/docs/query-data)
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/develop/develop-drill/002-patch-tool.md
----------------------------------------------------------------------
diff --git a/_docs/develop/develop-drill/002-patch-tool.md b/_docs/develop/develop-drill/002-patch-tool.md
new file mode 100644
index 0000000..28c8a54
--- /dev/null
+++ b/_docs/develop/develop-drill/002-patch-tool.md
@@ -0,0 +1,160 @@
+---
+title: "Drill Patch Review Tool"
+parent: "Develop Drill"
+---
+  * Drill JIRA and Reviewboard script
+    * 1\. Setup
+    * 2\. Usage
+    * 3\. Upload patch
+    * 4\. Update patch
+  * JIRA command line tool
+    * 1\. Download the JIRA command line package
+    * 2\. Configure JIRA username and password
+  * Reviewboard
+    * 1\. Install the post-review tool
+    * 2\. Configure Stuff
+  * FAQ
+    * When I run the script, it throws the following error and exits
+    * When I run the script, it throws the following error and exits
+
+### Drill JIRA and Reviewboard script
+
+#### 1\. Setup
+
+  1. Follow instructions [here](/docs/drill-patch-review-tool#jira-command-line-tool) to setup the jira-python package
+  2. Follow instructions [here](/docs/drill-patch-review-tool#reviewboard) to setup the reviewboard python tools
+  3. Install the argparse module 
+  
+        On Linux -> sudo yum install python-argparse
+        On Mac -> sudo easy_install argparse
+
+#### 2\. Usage
+
+	nnarkhed-mn: nnarkhed$ python drill-patch-review.py --help
+	usage: drill-patch-review.py [-h] -b BRANCH -j JIRA [-s SUMMARY]
+	                             [-d DESCRIPTION] [-r REVIEWBOARD] [-t TESTING]
+	                             [-v VERSION] [-db] -rbu REVIEWBOARDUSER -rbp REVIEWBOARDPASSWORD
+	 
+	Drill patch review tool
+	 
+	optional arguments:
+	  -h, --help            show this help message and exit
+	  -b BRANCH, --branch BRANCH
+	                        Tracking branch to create diff against
+	  -j JIRA, --jira JIRA  JIRA corresponding to the reviewboard
+	  -s SUMMARY, --summary SUMMARY
+	                        Summary for the reviewboard
+	  -d DESCRIPTION, --description DESCRIPTION
+	                        Description for reviewboard
+	  -r REVIEWBOARD, --rb REVIEWBOARD
+	                        Review board that needs to be updated
+	  -t TESTING, --testing-done TESTING
+	                        Text for the Testing Done section of the reviewboard
+	  -v VERSION, --version VERSION
+	                        Version of the patch
+	  -db, --debug          Enable debug mode
+	  -rbu, --reviewboard-user Reviewboard user name
+	  -rbp, --reviewboard-password Reviewboard password
+
+#### 3\. Upload patch
+
+  1. Specify the branch against which the patch should be created (-b)
+  2. Specify the corresponding JIRA (-j)
+  3. Specify an **optional** summary (-s) and description (-d) for the reviewboard
+
+Example:
+
+    python drill-patch-review.py -b origin/master -j DRILL-241 -rbu tnachen -rbp password
+
+#### 4\. Update patch
+
+  1. Specify the branch against which the patch should be created (-b)
+  2. Specify the corresponding JIRA (--jira)
+  3. Specify the rb to be updated (-r)
+  4. Specify an **optional** summary (-s) and description (-d) for the reviewboard, if you want to update it
+  5. Specify an **optional** version of the patch. This will be appended to the jira to create a file named JIRA-<version>.patch. The purpose is to be able to upload multiple patches to the JIRA. This has no bearing on the reviewboard update.
+
+Example:
+
+    python drill-patch-review.py -b origin/master -j DRILL-241 -r 14081 rbp tnachen -rbp password
+
+### JIRA command line tool
+
+#### 1\. Download the JIRA command line package
+
+Install the jira-python package.
+
+    sudo easy_install jira-python
+
+#### 2\. Configure JIRA username and password
+
+Include a jira.ini file in your $HOME directory that contains your Apache JIRA
+username and password.
+
+	nnarkhed-mn:~ nnarkhed$ cat ~/jira.ini
+	user=nehanarkhede
+	password=***********
+
+### Reviewboard
+
+This is a quick tutorial on using [Review Board](https://reviews.apache.org)
+with Drill.
+
+#### 1\. Install the post-review tool
+
+If you are on RHEL, Fedora or CentOS, follow these steps:
+
+	sudo yum install python-setuptools
+	sudo easy_install -U RBTools
+
+If you are on Mac, follow these steps:
+
+	sudo easy_install -U setuptools
+	sudo easy_install -U RBTools
+
+For other platforms, follow the [instructions](http://www.reviewboard.org/docs/manual/dev/users/tools/post-review/) to
+setup the post-review tool.
+
+#### 2\. Configure Stuff
+
+Then you need to configure a few things to make it work.
+
+First set the review board url to use. You can do this from in git:
+
+    git config reviewboard.url https://reviews.apache.org
+
+If you checked out using the git wip http url that confusingly won't work with
+review board. So you need to configure an override to use the non-http url.
+You can do this by adding a config file like this:
+
+	jkreps$ cat ~/.reviewboardrc
+	REPOSITORY = 'git://git.apache.org/incubator-drill.git'
+	TARGET_GROUPS = 'drill-git'
+GUESS_FIELDS = True
+
+
+
+### FAQ
+
+#### When I run the script, it throws the following error and exits
+
+    nnarkhed$python drill-patch-review.py -b trunk -j DRILL-241
+    There don't seem to be any diffs
+
+There are two reasons for this:
+
+  * The code is not checked into your local branch
+  * The -b branch is not pointing to the remote branch. In the example above, "trunk" is specified as the branch, which is the local branch. The correct value for the -b (--branch) option is the remote branch. "git branch -r" gives the list of the remote branch names.
+
+#### When I run the script, it throws the following error and exits
+
+Error uploading diff
+ 
+Your review request still exists, but the diff is not attached.
+
+One of the most common root causes of this error are that the git remote
+branches are not up-to-date. Since the script already does that, it is
+probably due to some other problem. You can run the script with the --debug
+option that will make post-review run in the debug mode and list the root
+cause of the issue.
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_1.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_1.png b/_docs/img/image_1.png
new file mode 100755
index 0000000..d9e3733
Binary files /dev/null and b/_docs/img/image_1.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_10.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_10.png b/_docs/img/image_10.png
new file mode 100755
index 0000000..459e775
Binary files /dev/null and b/_docs/img/image_10.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_11.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_11.png b/_docs/img/image_11.png
new file mode 100755
index 0000000..88b56b0
Binary files /dev/null and b/_docs/img/image_11.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_12.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_12.png b/_docs/img/image_12.png
new file mode 100755
index 0000000..9431849
Binary files /dev/null and b/_docs/img/image_12.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_13.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_13.png b/_docs/img/image_13.png
new file mode 100755
index 0000000..315ee19
Binary files /dev/null and b/_docs/img/image_13.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_14.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_14.png b/_docs/img/image_14.png
new file mode 100755
index 0000000..40783fc
Binary files /dev/null and b/_docs/img/image_14.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_15.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_15.png b/_docs/img/image_15.png
new file mode 100755
index 0000000..f045b16
Binary files /dev/null and b/_docs/img/image_15.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_16.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_16.png b/_docs/img/image_16.png
new file mode 100755
index 0000000..92ca856
Binary files /dev/null and b/_docs/img/image_16.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_17.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_17.png b/_docs/img/image_17.png
new file mode 100755
index 0000000..59aeebb
Binary files /dev/null and b/_docs/img/image_17.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_2.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_2.png b/_docs/img/image_2.png
new file mode 100755
index 0000000..bdabf09
Binary files /dev/null and b/_docs/img/image_2.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_3.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_3.png b/_docs/img/image_3.png
new file mode 100755
index 0000000..d5cb6dd
Binary files /dev/null and b/_docs/img/image_3.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_4.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_4.png b/_docs/img/image_4.png
new file mode 100755
index 0000000..b8eb1d3
Binary files /dev/null and b/_docs/img/image_4.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_5.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_5.png b/_docs/img/image_5.png
new file mode 100755
index 0000000..5af7198
Binary files /dev/null and b/_docs/img/image_5.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_6.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_6.png b/_docs/img/image_6.png
new file mode 100755
index 0000000..912284c
Binary files /dev/null and b/_docs/img/image_6.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_7.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_7.png b/_docs/img/image_7.png
new file mode 100755
index 0000000..141a700
Binary files /dev/null and b/_docs/img/image_7.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_8.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_8.png b/_docs/img/image_8.png
new file mode 100755
index 0000000..2df48f1
Binary files /dev/null and b/_docs/img/image_8.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/img/image_9.png
----------------------------------------------------------------------
diff --git a/_docs/img/image_9.png b/_docs/img/image_9.png
new file mode 100755
index 0000000..ee5b0d4
Binary files /dev/null and b/_docs/img/image_9.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/001-drill-in-10.md
----------------------------------------------------------------------
diff --git a/_docs/install/001-drill-in-10.md b/_docs/install/001-drill-in-10.md
index 4c9454d..37b8bd0 100644
--- a/_docs/install/001-drill-in-10.md
+++ b/_docs/install/001-drill-in-10.md
@@ -2,8 +2,6 @@
 title: "Apache Drill in 10 Minutes"
 parent: "Install Drill"
 ---
-[Previous](/docs/install-drill)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/deploying-apache-drill-in-a-clustered-environment)
-
 * Objective
 * A Few Bits About Apache Drill
 * Process Overview

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/002-deploy.md
----------------------------------------------------------------------
diff --git a/_docs/install/002-deploy.md b/_docs/install/002-deploy.md
index 4a498c2..399414e 100644
--- a/_docs/install/002-deploy.md
+++ b/_docs/install/002-deploy.md
@@ -2,9 +2,6 @@
 title: "Deploying Apache Drill in a Clustered Environment"
 parent: "Install Drill"
 ---
-[Previous](/docs/apache-drill-in-10-minutes
-)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-drill-in-embedded-mode)
-
 ## Overview
 
 To run Drill in a clustered environment, complete the following steps:

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/003-install-embedded.md
----------------------------------------------------------------------
diff --git a/_docs/install/003-install-embedded.md b/_docs/install/003-install-embedded.md
index 6c3ffcd..645755a 100644
--- a/_docs/install/003-install-embedded.md
+++ b/_docs/install/003-install-embedded.md
@@ -2,8 +2,6 @@
 title: "Installing Drill in Embedded Mode"
 parent: "Install Drill"
 ---
-[Previous](/docs/deploying-apache-drill-in-a-clustered-environment)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-drill-on-linux)
-
 Installing Drill in embedded mode installs Drill locally on your machine.
 Embedded mode is a quick, easy way to install and try Drill without having to
 perform any configuration tasks. When you install Drill in embedded mode, the

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/004-install-distributed.md
----------------------------------------------------------------------
diff --git a/_docs/install/004-install-distributed.md b/_docs/install/004-install-distributed.md
index 04a7414..a47176f 100644
--- a/_docs/install/004-install-distributed.md
+++ b/_docs/install/004-install-distributed.md
@@ -2,8 +2,6 @@
 title: "Installing Drill in Distributed Mode"
 parent: "Install Drill"
 ---
-[Previous](/docs/installing-drill-on-windows)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/connect-to-a-data-source)
-
 You can install Apache Drill in distributed mode on one or multiple nodes to
 run it in a clustered environment.
 

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/install-embedded/001-install-linux.md
----------------------------------------------------------------------
diff --git a/_docs/install/install-embedded/001-install-linux.md b/_docs/install/install-embedded/001-install-linux.md
index 375a87a..589fa0f 100644
--- a/_docs/install/install-embedded/001-install-linux.md
+++ b/_docs/install/install-embedded/001-install-linux.md
@@ -2,8 +2,6 @@
 title: "Installing Drill on Linux"
 parent: "Installing Drill in Embedded Mode"
 ---
-[Previous](/docs/installing-drill-in-embedded-mode)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-drill-on-mac-os-x)
-
 Complete the following steps to install Apache Drill on a machine running
 Linux:
 

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/install-embedded/002-install-mac.md
----------------------------------------------------------------------
diff --git a/_docs/install/install-embedded/002-install-mac.md b/_docs/install/install-embedded/002-install-mac.md
index ed3907f..97ae775 100644
--- a/_docs/install/install-embedded/002-install-mac.md
+++ b/_docs/install/install-embedded/002-install-mac.md
@@ -2,8 +2,6 @@
 title: "Installing Drill on Mac OS X"
 parent: "Installing Drill in Embedded Mode"
 ---
-[Previous](/docs/installing-drill-on-linux)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-drill-on-windows)
-
 Complete the following steps to install Apache Drill on a machine running Mac
 OS X:
 

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/install/install-embedded/003-install-win.md
----------------------------------------------------------------------
diff --git a/_docs/install/install-embedded/003-install-win.md b/_docs/install/install-embedded/003-install-win.md
index 442551f..6c8272b 100644
--- a/_docs/install/install-embedded/003-install-win.md
+++ b/_docs/install/install-embedded/003-install-win.md
@@ -2,8 +2,6 @@
 title: "Installing Drill on Windows"
 parent: "Installing Drill in Embedded Mode"
 ---
-[Previous](/docs/installing-drill-on-mac-os-x)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-drill-in-distributed-mode)
-
 You can install Drill on Windows 7 or 8. To install Drill on Windows, you must
 have JDK 7, and you must set the `JAVA_HOME` path in the Windows Environment
 Variables. You must also have a utility, such as

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/001-jdbc-squirrel.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/001-jdbc-squirrel.md b/_docs/interfaces/001-jdbc-squirrel.md
index 61c3678..0fd14c0 100644
--- a/_docs/interfaces/001-jdbc-squirrel.md
+++ b/_docs/interfaces/001-jdbc-squirrel.md
@@ -2,8 +2,6 @@
 title: "Using the JDBC Driver"
 parent: "ODBC/JDBC Interfaces"
 ---
-[Previous](/docs/testing-the-odbc-connection-on-linux-and-mac-os-x)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/query-data)
-
 To use the JDBC Driver to access Drill through Squirrel, ensure that you meet the prerequisites and follow the steps in this section.
 ### Prerequisites
 

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/002-odbc-linux.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/002-odbc-linux.md b/_docs/interfaces/002-odbc-linux.md
index 48a2814..0ed1070 100644
--- a/_docs/interfaces/002-odbc-linux.md
+++ b/_docs/interfaces/002-odbc-linux.md
@@ -2,8 +2,6 @@
 title: "Using the MapR ODBC Driver on Linux and Mac OS X"
 parent: "ODBC/JDBC Interfaces"
 ---
-[Previous](/docs/using-the-jdbc-driver)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-the-mapr-drill-odbc-driver-on-linux)
-
 The MapR Drill ODBC driver provides BI tools access to Drill’s flexible query
 capabilities so you can quickly explore various data sources. Once you install
 the MapR Drill ODBC Driver, you can configure ODBC connections to access Drill

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/003-odbc-win.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/003-odbc-win.md b/_docs/interfaces/003-odbc-win.md
index 33d73dc..5105048 100644
--- a/_docs/interfaces/003-odbc-win.md
+++ b/_docs/interfaces/003-odbc-win.md
@@ -2,8 +2,6 @@
 title: "Using the MapR ODBC Driver on Windows"
 parent: "ODBC/JDBC Interfaces"
 ---
-[Previous](/docs/testing-the-odbc-connection-on-linux-and-mac-os-x)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/step-1-install-the-mapr-drill-odbc-driver-on-windows)
-
 The MapR Drill ODBC driver provides BI tools access to Drill’s flexible query
 capabilities so that users can quickly explore various data sources. The MapR
 Drill ODBC driver includes Drill Explorer, which is a simple user interface

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/004-microstrategy.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/004-microstrategy.md b/_docs/interfaces/004-microstrategy.md
new file mode 100644
index 0000000..340a451
--- /dev/null
+++ b/_docs/interfaces/004-microstrategy.md
@@ -0,0 +1,158 @@
+---
+title: "Using MicroStrategy Analytics with Apache Drill"
+parent: "ODBC/JDBC Interfaces"
+---
+Apache Drill is certified with the MicroStrategy Analytics Enterprise Platform™. You can connect MicroStrategy Analytics Enterprise to Apache Drill and explore multiple data formats instantly on Hadoop. Use the combined power of these tools to get direct access to semi-structured data without having to rely on IT teams for schema creation.
+
+Complete the following steps to use Apache Drill with MicroStrategy Analytics Enterprise:
+ 
+1.  Install the Drill ODBC driver from MapR.
+2.	Configure the MicroStrategy Drill Object.
+3.	Create the MicroStrategy database connection for Drill.
+4.	Query and analyze the data.
+
+----------
+
+
+### Step 1: Install and Configure the MapR Drill ODBC Driver 
+
+Drill uses standard ODBC connectivity to provide you with easy data exploration capabilities on complex, schema-less data sets. Verify that the ODBC driver version is compatible with the Apache Drill version that you use.
+ 
+Ideally, you should upgrade to Drill 0.7, which uses the v0.07.0.0608 version of the driver. Earlier versions of Drill use the v0.07.0.0600 ODBC driver version located at:
+
+http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/MapRDrill_odbc_v0.07.0.0600/
+
+Complete the following steps to install and configure the driver:
+
+1.	Download the driver from the following location: 
+
+    http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/MapRDrill_odbc_v0.07.0.0608/MapRDrillODBC32.msi
+
+    **Note:** Use the 32-bit Windows driver for MicroStrategy 9.4.1.
+
+2.	Complete steps 2-8 under *Installing the Driver* on the following page: 
+
+    https://cwiki.apache.org/confluence/display/DRILL/Using+the+MapR+ODBC+Driver+on+Windows
+3.	Complete the steps on the following page to configure the driver:
+
+    https://cwiki.apache.org/confluence/display/DRILL/Step+2.+Configure+ODBC+Connections+to+Drill+Data+Sources 
+
+    **Note:** Verify that you are using the 32-bit driver since both drivers can coexist on the same machine. 
+
+	a.	Verify the version number of the driver.
+
+    ![]({{ site.baseurl }}/docs/img/image_17.png)
+	 
+	b.	Click Test to verify that the ODBC configuration works before using it with MicroStrategy.
+
+    ![]({{ site.baseurl }}/docs/img/image_2.png)
+
+----------
+
+
+### Step 2: Install the Drill Object on MicroStrategy Analytics Enterprise 
+The steps listed in this section were created based on the MicroStrategy Technote for installing DBMS objects which you can reference at: 
+
+http://community.microstrategy.com/t5/Database/TN43537-How-to-install-DBMS-objects-provided-by-MicroStrategy/ta-p/193352
+
+
+Complete the following steps to install the Drill Object on MicroStrategy Analytics Enterprise:
+
+1. Obtain the Drill Object from MicroStrategy Technical Support. The Drill Object is contained in a file named `MapR_Drill.PDS`. When you get this file, store it locally in your Windows file system.
+2. Open MicroStrategy Developer. 
+3. Expand Administration, and open Configuration Manager.
+4. Select **Database Instances**.
+   ![]({{ site.baseurl }}/docs/img/image_3.png)
+5. Right-click in the area where the current database instances display. 
+   ![]({{ site.baseurl }}/docs/img/image_4.png)
+6. Select **New – Database Instance**. 
+7. Once the Database Instances window opens, select **Upgrade**.
+   ![]({{ site.baseurl }}/docs/img/image_5.png)
+8. Enter the path and file name for the Drill Object file in the DB types script file field. Alternatively, you can use the browse button next to the field to search for the file. 
+   ![]({{ site.baseurl }}/docs/img/image_6.png)
+9.  Click **Load**. 
+10.	Once loaded, select the MapR Drill database type in the left column.
+11.	Click **>** to load MapR Drill into **Existing database types**. 
+12.	Click **OK** to save the database type.
+13.	Restart MicroStrategy Intelligence Server if it is used for the project source.
+   ![]({{ site.baseurl }}/docs/img/image_7.png)
+
+MicroStrategy Analytics Enterprise can now access Apache Drill.
+
+
+----------
+
+### Step 3: Create the MicroStrategy database connection for Apache Drill
+Complete the following steps to use the Database Instance Wizard to create the MicroStrategy database connection for Apache Drill:
+
+1. In MicroStrategy  Developer, select **Administration > Database Instance Wizard**.
+   ![]({{ site.baseurl }}/docs/img/image_8.png)
+2. Enter a name for the database, and select **MapR Drill** as the Database type from the drop-down menu.
+   ![]({{ site.baseurl }}/docs/img/image_9.png)
+3. Click **Next**. 
+4. Select the ODBC DSN that you configured with the ODBC Administrator.
+   ![]({{ site.baseurl }}/docs/img/image_10.png)
+5. Provide the login information for the connection and then click **Finish**.
+
+You can now use MicroStrategy Analytics Enterprise to access Drill as a database instance. 
+
+----------
+
+
+### Step 4: Query and Analyze the Data
+This step includes an example scenario that shows you how to use MicroStrategy, with Drill as the database instance, to analyze Twitter data stored as complex JSON documents. 
+
+####Scenario
+The Drill distributed file system plugin is configured to read Twitter data in a directory structure. A view is created in Drill to capture the most relevant maps and nested maps and arrays for the Twitter JSON documents. Refer to the following page for more information about how to configure and use Drill to work with complex data:
+
+https://cwiki.apache.org/confluence/display/DRILL/Query+Data
+
+####Part 1: Create a Project
+Complete the following steps to create a project:
+
+1. In MicroStrategy Developer, use the Project Creation Assistant to create a new project.
+   ![]({{ site.baseurl }}/docs/img/image_11.png)
+2.  Once the Assistant starts, click **Create Project**, and enter a name for the new project. 
+3.	Click **OK**. 
+4.	Click **Select tables from the Warehouse Catalog**. 
+5.	Select the Drill database instance connection from the drop down list, and click **OK**.	MicroStrategy queries Drill and displays all of the available tables and views.
+   ![]({{ site.baseurl }}/docs/img/image_12.png)
+6.	Select the two views created for the Twitter Data.
+7.	Use **>** to move the views to **Tables being used in the project**. 
+8.	Click **Save and Close**.
+9.	Click **OK**. The new project is created in MicroStrategy Developer. 
+
+####Part 2: Create a Freeform Report to Analyze Data
+Complete the following steps to create a Freeform Report and analyze data:
+
+1.	In Developer, open the Project and then open Public Objects.
+2.	Click **Reports**.
+3.	Right-click in the pane on the right, and select **New > Report**.
+   ![]({{ site.baseurl }}/docs/img/image_13.png)
+4.	Click the **Freeform Soures** tab, and select the Drill data source.
+   ![]({{ site.baseurl }}/docs/img/image_14.png)
+5.	Verify that **Create Freeform SQL Report** is selected, and click **OK**. This allows you to enter a quick query to gather data. The Freeform SQL Editor window appears.
+   ![]({{ site.baseurl }}/docs/img/image_15.png)
+6.	Enter a SQL query in the field provided. Attributes specified display. 
+In this scenario, a simple query that selects and groups the tweet source and counts the number of times the same source appeared in a day is entered. The tweet source was added as a text metric and the count as a number. 
+7.	Click **Data/Run Report** to run the query. A bar chart displays the output.
+   ![]({{ site.baseurl }}/docs/img/image_16.png)
+
+You can see that there are three major sources for the captured tweets. You can change the view to tabular format and apply a filter to see that iPhone, Android, and Web Client are the three major sources of tweets for this specific data set.
+![]({{ site.baseurl }}/docs/img/image_17.png)
+
+In this scenario, you learned how to configure MicroStrategy Analytics Enterprise to work with Apache Drill. 
+
+----------
+
+### Certification Links
+
+MicroStrategy announced post certification of Drill 0.6 and 0.7 with MicroStrategy Analytics Enterprise 9.4.1
+
+
+http://community.microstrategy.com/t5/Database/TN225724-Post-Certification-of-MapR-Drill-0-6-and-0-7-with/ta-p/225724
+
+http://community.microstrategy.com/t5/Release-Notes/TN231092-Certified-Database-and-ODBC-configurations-for/ta-p/231092
+
+http://community.microstrategy.com/t5/Release-Notes/TN231094-Certified-Database-and-ODBC-configurations-for/ta-p/231094   
+

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/odbc-linux/001-install-odbc-linux.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/odbc-linux/001-install-odbc-linux.md b/_docs/interfaces/odbc-linux/001-install-odbc-linux.md
index 9c497f6..61644d8 100644
--- a/_docs/interfaces/odbc-linux/001-install-odbc-linux.md
+++ b/_docs/interfaces/odbc-linux/001-install-odbc-linux.md
@@ -2,8 +2,6 @@
 title: "Installing the MapR Drill ODBC Driver on Linux"
 parent: "Using the MapR ODBC Driver on Linux and Mac OS X"
 ---
-[Previous](/docs/using-the-mapr-odbc-driver-on-linux-and-mac-os-x)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/installing-the-mapr-drill-odbc-driver-on-mac-os-x)
-
 Install the MapR Drill ODBC Driver on the machine from which you connect to
 the Drill service. You can install the 32- or 64-bit driver on Linux. Install
 the version of the driver that matches the architecture of the client

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/odbc-linux/002-install-odbc-mac.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/odbc-linux/002-install-odbc-mac.md b/_docs/interfaces/odbc-linux/002-install-odbc-mac.md
index 11c52d4..77b8e1b 100644
--- a/_docs/interfaces/odbc-linux/002-install-odbc-mac.md
+++ b/_docs/interfaces/odbc-linux/002-install-odbc-mac.md
@@ -2,8 +2,6 @@
 title: "Installing the MapR Drill ODBC Driver on Mac OS X"
 parent: "Using the MapR ODBC Driver on Linux and Mac OS X"
 ---
-[Previous](/docs/installing-the-mapr-drill-odbc-driver-on-linux)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/configuring-odbc-connections-for-linux-and-mac-os-x)
-
 Install the MapR Drill ODBC Driver on the machine from which you connect to
 the Drill service.
 

http://git-wip-us.apache.org/repos/asf/drill/blob/85e05623/_docs/interfaces/odbc-linux/003-odbc-connections-linux.md
----------------------------------------------------------------------
diff --git a/_docs/interfaces/odbc-linux/003-odbc-connections-linux.md b/_docs/interfaces/odbc-linux/003-odbc-connections-linux.md
index 0cd194e..f6f276e 100644
--- a/_docs/interfaces/odbc-linux/003-odbc-connections-linux.md
+++ b/_docs/interfaces/odbc-linux/003-odbc-connections-linux.md
@@ -2,8 +2,6 @@
 title: "Configuring ODBC Connections for Linux and Mac OS X"
 parent: "Using the MapR ODBC Driver on Linux and Mac OS X"
 ---
-[Previous](/docs/installing-the-mapr-drill-odbc-driver-on-mac-os-x)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Back to Table of Contents](/docs)<code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code>[Next](/docs/driver-configuration-options)
-
 ODBC driver managers use configuration files to define and configure ODBC data
 sources and drivers. Before you connect to Drill through an ODBC client tool
 on Linux or Mac OS X, you must update the following configuration files:


Mime
View raw message