carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chenliang...@apache.org
Subject incubator-carbondata-site git commit: sync up documents
Date Thu, 23 Mar 2017 05:37:17 GMT
Repository: incubator-carbondata-site
Updated Branches:
  refs/heads/asf-site fb599a3c1 -> bc1361dda


sync up documents


Project: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/commit/bc1361dd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/tree/bc1361dd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/diff/bc1361dd

Branch: refs/heads/asf-site
Commit: bc1361dda184d6ab549f42c3de7c19469d7c6185
Parents: fb599a3
Author: chenliang613 <chenliang613@huawei.com>
Authored: Thu Mar 23 11:05:43 2017 +0530
Committer: chenliang613 <chenliang613@huawei.com>
Committed: Thu Mar 23 11:05:43 2017 +0530

----------------------------------------------------------------------
 .../How-to-contribute-to-Apache-CarbonData.md   | 193 ++++++++
 src/site/markdown/configuration-parameters.md   |   3 +-
 .../markdown/file-structure-of-carbondata.md    |  21 +-
 src/site/markdown/images/CarbonData_logo.png    | Bin
 .../images/carbon_data_file_structure_new.png   | Bin 78374 -> 9477 bytes
 .../markdown/images/carbon_data_format_new.png  | Bin 73708 -> 35510 bytes
 .../markdown/images/query_failure_beeline.png   | Bin 48077 -> 0 bytes
 .../markdown/images/query_failure_issue.png     | Bin 316072 -> 0 bytes
 .../images/query_failure_job_details.png        | Bin 113176 -> 0 bytes
 src/site/markdown/images/query_failure_logs.png | Bin 441678 -> 0 bytes
 .../markdown/images/query_failure_procedure.png | Bin 80817 -> 0 bytes
 .../markdown/images/query_failure_spark_ui.png  | Bin 212907 -> 0 bytes
 src/site/markdown/quick-start-guide.md          |   9 +-
 src/site/markdown/release-guide.md              | 482 +++++++++++++++++++
 .../supported-data-types-in-carbondata.md       |  16 +-
 15 files changed, 707 insertions(+), 17 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/How-to-contribute-to-Apache-CarbonData.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/How-to-contribute-to-Apache-CarbonData.md b/src/site/markdown/How-to-contribute-to-Apache-CarbonData.md
new file mode 100644
index 0000000..7d0136e
--- /dev/null
+++ b/src/site/markdown/How-to-contribute-to-Apache-CarbonData.md
@@ -0,0 +1,193 @@
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+-->
+
+# How to contribute to Apache CarbonData
+
+The Apache CarbonData community welcomes all kinds of contributions from anyone with a passion
for
+faster data format! Apache CarbonData is a new file format for faster interactive query using
+advanced columnar storage, index, compression and encoding techniques to improve computing
+efficiency,in turn it will help speedup queries an order of magnitude faster over PetaBytes
of data.
+
+We use a review-then-commit workflow in CarbonData for all contributions.
+
+* Engage -> Design -> Code -> Review -> Commit
+
+## Engage
+
+### Mailing list(s)
+
+We discuss design and implementation issues on dev@carbondata.incubator.apache.org Join by
+emailing dev-subscribe@carbondata.incubator.apache.org
+
+### Apache JIRA
+
+We use [Apache JIRA](https://issues.apache.org/jira/browse/CARBONDATA) as an issue tracking
and
+project management tool, as well as a way to communicate among a very diverse and distributed
set
+of contributors. To be able to gather feedback, avoid frustration, and avoid duplicated efforts
all
+CarbonData-related work should be tracked there.
+
+If you do not already have an Apache JIRA account, sign up [here](https://issues.apache.org/jira/).
+
+If a quick search doesn’t turn up an existing JIRA issue for the work you want to contribute,
+create it. Please discuss your proposal with a committer or the component lead in JIRA or,
+alternatively, on the developer mailing list(dev@carbondata.incubator.apache.org).
+
+If there’s an existing JIRA issue for your intended contribution, please comment about
your
+intended work. Once the work is understood, a committer will assign the issue to you.
+(If you don’t have a JIRA role yet, you’ll be added to the “contributor” role.) If
an issue is
+currently assigned, please check with the current assignee before reassigning.
+
+For moderate or large contributions, you should not start coding or writing a design doc
unless
+there is a corresponding JIRA issue assigned to you for that work. Simple changes,
+like fixing typos, do not require an associated issue.
+
+### Design
+
+To avoid potential frustration during the code review cycle, we encourage you to clearly
scope and
+design non-trivial contributions with the CarbonData community before you start coding.
+
+Generally, the JIRA issue is the best place to gather relevant design docs, comments, or
references.
+It’s great to explicitly include relevant stakeholders early in the conversation. For designs
that
+may be generally interesting, we also encourage conversations on the developer’s mailing
list.
+
+### Code
+
+We use GitHub’s pull request functionality to review proposed code changes.
+If you do not already have a personal GitHub account, sign up [here](https://github.com).
+
+### Git config
+
+Ensure to finish the below config(user.email, user.name) before starting PR works.
+```
+$ git config --global user.email "you@example.com"
+$ git config --global user.name "Your Name"
+```
+
+#### Fork the repository on GitHub
+
+Go to the [Apache CarbonData GitHub mirror](https://github.com/apache/incubator-carbondata)
and
+fork the repository to your own private account.
+This will be your private workspace for staging changes.
+
+#### Clone the repository locally
+
+You are now ready to create the development environment on your local machine.
+Clone CarbonData’s read-only GitHub mirror.
+```
+$ git clone https://github.com/apache/incubator-carbondata.git
+$ cd incubator-carbondata
+```
+Add your forked repository as an additional Git remote, where you’ll push your changes.
+```
+$ git remote add <GitHub_user> https://github.com/<GitHub_user>/incubator-carbondata.git
+```
+You are now ready to start developing!
+
+#### Create a branch in your fork
+
+You’ll work on your contribution in a branch in your own (forked) repository. Create a
local branch,
+initialized with the state of the branch you expect your changes to be merged into.
+Keep in mind that we use several branches, including master, feature-specific, and
+release-specific branches. If you are unsure, initialize with the state of the master branch.
+```
+$ git fetch --all
+$ git checkout -b <my-branch> origin/master
+```
+At this point, you can start making and committing changes to this branch in a standard way.
+
+#### Syncing and pushing your branch
+
+Periodically while you work, and certainly before submitting a pull request, you should update
+your branch with the most recent changes to the target branch.
+```
+$ git pull --rebase
+```
+Remember to always use --rebase parameter to avoid extraneous merge commits.
+
+To push your local, committed changes to your (forked) repository on GitHub, run:
+```
+$ git push <GitHub_user> <my-branch>
+```
+#### Testing
+
+All code should have appropriate unit testing coverage. New code should have new tests in
the
+same contribution. Bug fixes should include a regression test to prevent the issue from reoccurring.
+
+For contributions to the Java code, run unit tests locally via Maven.
+```
+$ mvn clean verify
+```
+
+### Review
+
+Once the initial code is complete and the tests pass, it’s time to start the code review
process.
+We review and discuss all code, no matter who authors it. It’s a great way to build community,
+since you can learn from other developers, and they become familiar with your contribution.
+It also builds a strong project by encouraging a high quality bar and keeping code consistent
+throughout the project.
+
+#### Create a pull request
+
+Organize your commits to make your reviewer’s job easier. Use the following command to
+re-order, squash, edit, or change description of individual commits.
+```
+$ git rebase -i origin/master
+```
+Navigate to the CarbonData GitHub mirror to create a pull request. The title of the pull
request
+should be strictly in the following format:
+```
+[CARBONDATA-issue number>] Title of the pull request
+```
+Please include a descriptive pull request message to help make the reviewer’s job easier.
+
+If you know a good committer to review your pull request, please make a comment like the
following.
+If not, don’t worry, a committer will pick it up.
+```
+Hi @<committer/reviewer name>, can you please take a look?
+```
+
+#### Code Review and Revision
+
+During the code review process, don’t rebase your branch or otherwise modify published
commits,
+since this can remove existing comment history and be confusing to the reviewer,
+When you make a revision, always push it in a new commit.
+
+Our GitHub mirror automatically provides pre-commit testing coverage using Jenkins.
+Please make sure those tests pass,the contribution cannot be merged otherwise.
+
+#### LGTM
+Once the reviewer is happy with the change, they’ll respond with an LGTM (“looks good
to me!”).
+At this point, the committer will take over, possibly make some additional touch ups,
+and merge your changes into the codebase.
+
+In the case both the author and the reviewer are committers, either can merge the pull request.
+Just be sure to communicate clearly whose responsibility it is in this particular case.
+
+Thank you for your contribution to Apache CarbonData!
+
+#### Deleting your branch(optional)
+Once the pull request is merged into the Apache CarbonData repository, you can safely delete
the
+branch locally and purge it from your forked repository.
+
+From another local branch, run:
+```
+$ git fetch --all
+$ git branch -d <my-branch>
+$ git push <GitHub_user> --delete <my-branch>
+```

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/configuration-parameters.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/configuration-parameters.md b/src/site/markdown/configuration-parameters.md
index 75001be..774734a 100644
--- a/src/site/markdown/configuration-parameters.md
+++ b/src/site/markdown/configuration-parameters.md
@@ -132,7 +132,7 @@ This section provides the details of all the configurations required for
CarbonD
 | Parameter | Default Value | Description |
 |---------------------------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | high.cardinality.identify.enable | true | If the parameter is true, the high cardinality
columns of the dictionary code are automatically recognized and these columns will not be
used as global dictionary encoding. If the parameter is false, all dictionary encoding columns
are used as dictionary encoding. The high cardinality column must meet the following requirements:
value of cardinality > configured value of high.cardinalityEqually, the value of cardinality
is higher than the threshold.value of cardinality/ row number x 100 > configured value
of high.cardinality.row.count.percentageEqually, the ratio of the cardinality value to data
row number is higher than the configured percentage. |
-| high.cardinality.threshold | 1000000 | It is a threshold to identify high cardinality of
the columns.If the value of columns' cardinality > the configured value, then the columns
are excluded from dictionary encoding. |
+| high.cardinality.threshold | 1000000 | high.cardinality.threshold | 1000000 | It is a threshold
to identify high cardinality of the columns.If the value of columns' cardinality > the
configured value, then the columns are excluded from dictionary encoding. |
 | high.cardinality.row.count.percentage | 80 | Percentage to identify whether column cardinality
is more than configured percent of total row count.Configuration value formula:Value of cardinality/
row number x 100 > configured value of high.cardinality.row.count.percentageThe value of
the parameter must be larger than 0. |
 | carbon.cutOffTimestamp | 1970-01-01 05:30:00 | Sets the start date for calculating the
timestamp. Java counts the number of milliseconds from start of "1970-01-01 00:00:00". This
property is used to customize the start of position. For example "2000-01-01 00:00:00". The
date must be in the form "carbon.timestamp.format". NOTE: The CarbonData supports data store
up to 68 years from the cut-off time defined. For example, if the cut-off time is 1970-01-01
05:30:00, then the data can be stored up to 2038-01-01 05:30:00. |
 | carbon.timegranularity | SECOND | The property used to set the data granularity level DAY,
HOUR, MINUTE, or SECOND. |
@@ -145,4 +145,5 @@ This section provides the details of all the configurations required for
CarbonD
 | spark.driver.memory | 1g | Amount of memory to be used by the driver process. |
 | spark.executor.memory | 1g | Amount of memory to be used per executor process. |
 | spark.sql.bigdata.register.analyseRule | org.apache.spark.sql.hive.acl.CarbonAccessControlRules
| CarbonAccessControlRules need to be set for enabling Access Control. |
+   
  
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/file-structure-of-carbondata.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/file-structure-of-carbondata.md b/src/site/markdown/file-structure-of-carbondata.md
index 63e34ec..bfbcee4 100644
--- a/src/site/markdown/file-structure-of-carbondata.md
+++ b/src/site/markdown/file-structure-of-carbondata.md
@@ -17,20 +17,29 @@
     under the License.
 -->
 
-#  CarbonData File Structure
+# CarbonData File Structure
 
-CarbonData files contain groups of data called blocklets, along with all required information
like schema, offsets and indices etc, in a file footer, co-located in HDFS.
+CarbonData files contain groups of data called blocklets, along with all required information
like schema, offsets and indices etc, in a file header and footer, co-located in HDFS.
 
 The file footer can be read once to build the indices in memory, which can be utilized for
optimizing the scans and processing for all subsequent queries.
 
-Each blocklet in the file is further divided into chunks of data called data chunks. Each
data chunk is organized either in columnar format or row format, and stores the data of either
a single column or a set of columns. All blocklets in a file contain the same number and type
of data chunks.
+### Understanding CarbonData File Structure
+* Block : It would be as same as HDFS block, CarbonData creates one file for each data block,
user can specify TABLE_BLOCKSIZE during creation table. Each file contains File Header, Blocklets
and File Footer. 
 
 ![CarbonData File Structure](../docs/images/carbon_data_file_structure_new.png?raw=true)
 
-Each data chunk contains multiple groups of data called as pages. There are three types of
pages.
+* File Header : It contains CarbonData file version number, list of column schema and schema
updation timestamp.
+* File Footer : it contains Number of rows, segmentinfo ,all blocklets’ info and index,
you can find the detail from the below diagram.
+* Blocklet : Rows are grouped to form a blocklet, the size of the blocklet is configurable
and default size is 64MB, Blocklet contains Column Page groups for each column.
+* Column Page Group : Data of one column and it is further divided to pages, it is guaranteed
to be contiguous in file.
+* Page : It has the data of one column and the number of row is fixed to 32000 size. 
 
-* Data Page: Contains the encoded data of a column/group of columns.
+![CarbonData File Format](../docs/images/carbon_data_format_new.png?raw=true)
+
+### Each page contains three types of data
+* Data Page: Contains the encoded data of a column of columns.
 * Row ID Page (optional): Contains the row ID mappings used when the data page is stored
as an inverted index.
 * RLE Page (optional): Contains additional metadata used when the data page is RLE coded.
 
-![CarbonData File Format](../docs/images/carbon_data_format_new.png?raw=true)
+
+

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/CarbonData_logo.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/CarbonData_logo.png b/src/site/markdown/images/CarbonData_logo.png
old mode 100755
new mode 100644

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/carbon_data_file_structure_new.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/carbon_data_file_structure_new.png b/src/site/markdown/images/carbon_data_file_structure_new.png
old mode 100755
new mode 100644
index 3f9241b..1c6f22b
Binary files a/src/site/markdown/images/carbon_data_file_structure_new.png and b/src/site/markdown/images/carbon_data_file_structure_new.png
differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/carbon_data_format_new.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/carbon_data_format_new.png b/src/site/markdown/images/carbon_data_format_new.png
old mode 100755
new mode 100644
index 9d0b194..f0fc553
Binary files a/src/site/markdown/images/carbon_data_format_new.png and b/src/site/markdown/images/carbon_data_format_new.png
differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/query_failure_beeline.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_beeline.png b/src/site/markdown/images/query_failure_beeline.png
deleted file mode 100644
index e4ec22b..0000000
Binary files a/src/site/markdown/images/query_failure_beeline.png and /dev/null differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/query_failure_issue.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_issue.png b/src/site/markdown/images/query_failure_issue.png
deleted file mode 100644
index 87270d2..0000000
Binary files a/src/site/markdown/images/query_failure_issue.png and /dev/null differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/query_failure_job_details.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_job_details.png b/src/site/markdown/images/query_failure_job_details.png
deleted file mode 100644
index 26e607d..0000000
Binary files a/src/site/markdown/images/query_failure_job_details.png and /dev/null differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/query_failure_logs.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_logs.png b/src/site/markdown/images/query_failure_logs.png
deleted file mode 100644
index 8fbdfa6..0000000
Binary files a/src/site/markdown/images/query_failure_logs.png and /dev/null differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/query_failure_procedure.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_procedure.png b/src/site/markdown/images/query_failure_procedure.png
deleted file mode 100644
index 9d2c81f..0000000
Binary files a/src/site/markdown/images/query_failure_procedure.png and /dev/null differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/images/query_failure_spark_ui.png
----------------------------------------------------------------------
diff --git a/src/site/markdown/images/query_failure_spark_ui.png b/src/site/markdown/images/query_failure_spark_ui.png
deleted file mode 100644
index 1802760..0000000
Binary files a/src/site/markdown/images/query_failure_spark_ui.png and /dev/null differ

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/quick-start-guide.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/quick-start-guide.md b/src/site/markdown/quick-start-guide.md
index c298b0d..c29a8d3 100644
--- a/src/site/markdown/quick-start-guide.md
+++ b/src/site/markdown/quick-start-guide.md
@@ -34,7 +34,9 @@ This tutorial provides a quick introduction to using CarbonData.
   EOF
   ```
 
-## Interactive Analysis with Spark Shell Version 2.1
+## Interactive Analysis with Spark Shell
+
+## Version 2.1
 
 Apache Spark Shell provides a simple way to learn the API, as well as a powerful tool to
analyze data interactively. Please visit [Apache Spark Documentation](http://spark.apache.org/docs/latest/)
for more details on Spark shell.
 
@@ -46,7 +48,7 @@ Start Spark shell by running the following command in the Spark directory:
 ./bin/spark-shell --jars <carbondata assembly jar path>
 ```
 
-In this shell, SparkSession is readily available as 'spark' and Spark context is readily
available as 'sc'.
+In this shell, SparkSession is readily available as `spark` and Spark context is readily
available as `sc`.
 
 In order to create a CarbonSession we will have to configure it explicitly in the following
manner :
 
@@ -87,7 +89,8 @@ scala>carbon.sql("SELECT * FROM test_table").show()
 scala>carbon.sql("SELECT city, avg(age), sum(age) FROM test_table GROUP BY city").show()
 ```
 
-## Interactive Analysis with Spark Shell Version 1.6
+## Interactive Analysis with Spark Shell
+## Version 1.6
 
 #### Basics
 

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/release-guide.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/release-guide.md b/src/site/markdown/release-guide.md
new file mode 100644
index 0000000..50a0e8a
--- /dev/null
+++ b/src/site/markdown/release-guide.md
@@ -0,0 +1,482 @@
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+-->
+
+# Apache CarbonData Release Guide
+
+Apache CarbonData periodically declares and publishes releases.
+
+Each release is executed by a _Release Manager_, who is selected among the CarbonData committers.
+ This document describes the process that the Release Manager follows to perform a release.
Any 
+ changes to this process should be discussed and adopted on the 
+ [dev@ mailing list](mailto:dev@carbondata.incubator.apache.org).
+ 
+Please remember that publishing software has legal consequences. This guide complements the

+foundation-wide [Product Release Policy](http://www.apache.org/dev/release.html) and [Release

+Distribution Policy](http://www.apache.org/dev/release-distribution).
+
+## Decide to release
+
+Deciding to release and selecting a Release Manager is the first step of the release process.

+This is a consensus-based decision of the entire community.
+
+Anybody can propose a release on the dev@ mailing list, giving a solid argument and nominating
a 
+committer as the Release Manager (including themselves). There's no formal process, no vote

+requirements, and no timing requirements. Any objections should be resolved by consensus
before 
+starting the release.
+
+_Checklist to proceed to next step:_
+
+1. Community agrees to release
+2. Community selects a Release Manager
+
+## Prepare for the release
+
+Before your first release, you should perform one-time configuration steps. This will set
up your
+ security keys for signing the artifacts and access release repository.
+ 
+To prepare for each release, you should audit the project status in the Jira, and do necessary

+bookkeeping. Finally, you should tag a release.
+
+### One-time setup instructions
+
+#### GPG Key
+
+You need to have a GPG key to sign the release artifacts. Please be aware of the ASF-wide

+[release signing guidelines](https://www.apache.org/dev/release-signing.html). If you don't
have 
+a GPG key associated with your Apache account, please create one according to the guidelines.
+
+Determine your Apache GPG key and key ID, as follows:
+
+```
+gpg --list-keys
+```
+
+This will list your GPG keys. One of these should reflect your Apache account, for exemple:
+
+```
+pub   2048R/845E6689 2016-02-23
+uid                  Nomen Nescio <anonymous@apache.org>
+sub   2048R/BA4D50BE 2016-02-23
+```
+
+Here, the key ID is the 8-digit hex string in the `pub` line: `845E6689`.
+
+Now, add your Apache GPG key to the CarbonData's `KEYS` file in `dev` and `release` repositories

+at `dist.apache.org`. Follow the instructions listed at the top of these files.
+ 
+Configure `git` to use this key when signing code by giving it your key ID, as follows:
+
+```
+git config --global user.signingkey 845E6689
+```
+
+You may drop the `--global` option if you'd prefer to use this key for the current repository
only.
+
+You may wish to start `gpg-agent` to unlock your GPG key only once using your passphrase.

+Otherwise, you may need to enter this passphrase several times. The setup of `gpg-agent`
varies 
+based on operating system, but may be something like this:
+
+```
+eval $(gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info)
+export GPG_TTY=$(tty)
+export GPG_AGENT_INFO
+```
+
+#### Access to Apache Nexus
+
+Configure access to the [Apache Nexus repository](https://repository.apache.org), used for

+staging repository and promote the artifacts to Maven Central.
+
+1. You log in with your Apache account.
+2. Confirm you have appropriate access by finding `org.apache.carbondata` under `Staging
Profiles`.
+3. Navigate to your `Profile` (top right dropdown menu of the page).
+4. Choose `User Token` from the dropdown, then click `Access User Token`. Copy a snippet
of the 
+Maven XML configuration block.
+5. Insert this snippet twice into your global Maven `settings.xml` file, typically `${HOME]/
+.m2/settings.xml`. The end result should look like this, where `TOKEN_NAME` and `TOKEN_PASSWORD`

+are your secret tokens:
+
+```
+ <settings>
+   <servers>
+     <server>
+       <id>apache.releases.https</id>
+       <username>TOKEN_NAME</username>
+       <password>TOKEN_PASSWORD</password>
+     </server>
+     <server>
+       <id>apache.snapshots.https</id>
+       <username>TOKEN_NAME</username>
+       <password>TOKEN_PASSWORD</password>
+     </server>
+   </servers>
+ </settings>
+```
+
+#### Create a new version in Jira
+
+When contributors resolve an issue in Jira, they are tagging it with a release that will
contain 
+their changes. With the release currently underway, new issues should be resolved against
a 
+subsequent future release. Therefore, you should create a release item for this subsequent

+release, as follows:
+
+1. In Jira, navigate to `CarbonData > Administration > Versions`.
+2. Add a new release: choose the next minor version number compared to the one currently

+underway, select today's date as the `Start Date`, and choose `Add`. 
+
+#### Triage release-blocking issues in Jira
+
+There could be outstanding release-blocking issues, which should be triaged before proceeding
to 
+build the release. We track them by assigning a specific `Fix Version` field even before
the 
+issue is resolved.
+
+The list of release-blocking issues is available at the [version status page](https://issues.apache.org/jira/browse/CARBONDATA/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel).

+Triage each unresolved issue with one of the following resolutions:
+
+* If the issue has been resolved and Jira was not updated, resolve it accordingly.
+* If the issue has not been resolved and it is acceptable to defer until the next release,
update
+ the `Fix Version` field to the new version you just created. Please consider discussing
this 
+ with stakeholders and the dev@ mailing list, as appropriate.
+* If the issue has not been resolved and it is not acceptable to release until it is fixed,
the 
+ release cannot proceed. Instead, work with the CarbonData community to resolve the issue.
+ 
+#### Review Release Notes in Jira
+
+Jira automatically generates Release Notes based on the `Fix Version` applied to the issues.

+Release Notes are intended for CarbonData users (not CarbonData committers/contributors).
You 
+should ensure that Release Notes are informative and useful.
+
+Open the release notes from the [version status page](https://issues.apache.org/jira/browse/CARBONDATA/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel)
+by choosing the release underway and clicking Release Notes.
+
+You should verify that the issues listed automatically by Jira are appropriate to appear
in the 
+Release Notes. Specifically, issues should:
+
+* Be appropriate classified as `Bug`, `New Feature`, `Improvement`, etc.
+* Represent noteworthy user-facing changes, such as new functionality, backward-incompatible

+changes, or performance improvements.
+* Have occurred since the previous release; an issue that was introduced and fixed between

+releases should not appear in the Release Notes.
+* Have an issue title that makes sense when read on its own.
+
+Adjust any of the above properties to the improve clarity and presentation of the Release
Notes.
+
+#### Verify that a Release Build works
+
+Run `mvn clean install -Prelease` to ensure that the build processes that are specific to
that 
+profile are in good shape.
+
+_Checklist to proceed to the next step:_
+
+1. Release Manager's GPG key is published to `dist.apache.org`.
+2. Release Manager's GPG key is configured in `git` configuration.
+3. Release Manager has `org.apache.carbondata` listed under `Staging Profiles` in Nexus.
+4. Release Manager's Nexus User Token is configured in `settings.xml`.
+5. Jira release item for the subsequent release has been created.
+6. There are no release blocking Jira issues.
+7. Release Notes in Jira have been audited and adjusted.
+
+### Build a release
+
+Use Maven release plugin to tag and build release artifacts, as follows:
+
+```
+mvn release:prepare
+```
+
+Use Maven release plugin to stage these artifacts on the Apache Nexus repository, as follows:
+
+```
+mvn release:perform
+```
+
+Review all staged artifacts. They should contain all relevant parts for each module, including

+`pom.xml`, jar, test jar, source, etc. Artifact names should follow 
+[the existing format](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.carbondata%22)
+in which artifact name mirrors directory structure. Carefully review any new artifacts.
+
+Close the staging repository on Nexus. When prompted for a description, enter "Apache CarbonData

+x.x.x release".
+
+### Stage source release on dist.apache.org
+
+Copy the source release to dev repository on `dist.apache.org`.
+
+1. If you have not already, check out the Incubator section of the `dev` repository on `dist
+.apache.org` via Subversion. In a fresh directory:
+
+```
+svn co https://dist.apache.org/repos/dist/dev/incubator/carbondata
+```
+
+2. Make a directory for the new release:
+
+```
+mkdir x.x.x
+```
+
+3. Copy the CarbonData source distribution, hash, and GPG signature:
+
+```
+cp apache-carbondata-x.x.x-source-release.zip x.x.x
+```
+
+4. Add and commit the files:
+
+```
+svn add x.x.x
+svn commit
+```
+
+5. Verify the files are [present](https://dist.apache.org/repos/dist/dev/incubator/carbondata).
+
+### Propose a pull request for website updates
+
+The final step of building a release candidate is to propose a website pull request.
+
+This pull request should update the following page with the new release:
+
+* `src/main/webapp/index.html`
+* `src/main/webapp/docs/latest/mainpage.html`
+
+_Checklist to proceed to the next step:_
+
+1. Maven artifacts deployed to the staging repository of 
+[repository.apache.org](https://repository.apache.org)
+2. Source distribution deployed to the dev repository of
+[dist.apache.org](https://dist.apache.org/repos/dist/dev/incubator/carbondata/)
+3. Website pull request to list the release.
+
+## Vote on the release candidate
+
+Once you have built and individually reviewed the release candidate, please share it for
the 
+community-wide review. Please review foundation-wide [voting guidelines](http://www.apache.org/foundation/voting.html)
+for more information.
+
+Start the review-and-vote thread on the dev@ mailing list. Here's an email template; please

+adjust as you see fit:
+
+```
+From: Release Manager
+To: dev@carbondata.incubator.apache.org
+Subject: [VOTE] Apache CarbonData Release x.x.x
+
+Hi everyone,
+Please review and vote on the release candidate for the version x.x.x, as follows:
+
+[ ] +1, Approve the release
+[ ] -1, Do not approve the release (please provide specific comments)
+
+The complete staging area is available for your review, which includes:
+* JIRA release notes [1],
+* the official Apache source release to be deployed to dist.apache.org [2], which is signed
with the key with fingerprint FFFFFFFF [3],
+* all artifacts to be deployed to the Maven Central Repository [4],
+* source code tag "x.x.x" [5],
+* website pull request listing the release [6].
+
+The vote will be open for at least 72 hours. It is adopted by majority approval, with at
least 3 PMC affirmative votes.
+
+Thanks,
+Release Manager
+
+[1] link
+[2] link
+[3] https://dist.apache.org/repos/dist/dist/incubator/carbondata/KEYS
+[4] link
+[5] link
+[6] link
+```
+
+If there are any issues found in the release candidate, reply on the vote thread to cancel
the vote.
+There’s no need to wait 72 hours. Proceed to the `Cancel a Release (Fix Issues)` step below
and 
+address the problem.
+However, some issues don’t require cancellation.
+For example, if an issue is found in the website pull request, just correct it on the spot
and the
+vote can continue as-is.
+
+If there are no issues, reply on the vote thread to close the voting. Then, tally the votes
in a
+separate email. Here’s an email template; please adjust as you see fit.
+
+```
+From: Release Manager
+To: dev@carbondata.incubator.apache.org
+Subject: [RESULT][VOTE] Apache CarbonData Release x.x.x
+
+I'm happy to announce that we have unanimously approved this release.
+
+There are XXX approving votes, XXX of which are binding:
+* approver 1
+* approver 2
+* approver 3
+* approver 4
+
+There are no disapproving votes.
+
+Thanks everyone!
+```
+
+While in incubation, the Apache Incubator PMC must also vote on each release, using the same

+process as above. Start the review and vote thread on the `general@incubator.apache.org`
list.
+
+```
+From: Release Manager
+To: general@incubator.apache.org
+Cc: dev@carbondata.incubator.apache.org
+Subject: [VOTE] Apache CarbonData release x.x.x-incubating
+
+Hi everyone,
+Please review and vote on the release candidate for the Apache CarbonData version x.x.x-incubating,
+ as follows:
+ 
+[ ] +1, Approve the release
+[ ] -1, Do not approve the release (please provide specific comments)
+
+The complete staging area is available for your review, which includes:
+* JIRA release notes [1],
+* the official Apache source release to be deployed to dist.apache.org [2],
+* all artifacts to be deployed to the Maven Central Repository [3],
+* source code tag "x.x.x" [4],
+* website pull request listing the release [5].
+
+The Apache CarbonData community has unanimously approved this release [6].
+
+As customary, the vote will be open for at least 72 hours. It is adopted by
+a majority approval with at least three PMC affirmative votes. If approved,
+we will proceed with the release.
+
+Thanks!
+
+[1] link
+[2] link
+[3] link
+[4] link
+[5] link
+[6] lists.apache.org permalink to the vote result thread, e.g.,  https://lists.apache.org/thread
+.html/32c991987e0abf2a09cd8afad472cf02e482af02ac35418ee8731940@%3Cdev.carbondata.apache.org%3E
+```
+
+If passed, close the voting and summarize the results:
+ 
+```
+From: Release Manager
+To: general@incubator.apache.org
+Cc: dev@carbondata.incubator.apache.org
+Subject: [RESULT][VOTE] Apache CarbonData release x.x.x-incubating
+
+There are XXX approving votes, all of which are binding:
+* approver 1
+* approver 2
+* approver 3
+* approver 4
+
+There are no disapproving votes.
+
+We'll proceed with this release as staged.
+
+Thanks everyone!
+```
+
+_Checklist to proceed to the final step:_
+
+1. Community votes to release the proposed release
+2. While in incubation, Apache Incubator PMC votes to release the proposed release
+
+## Cancel a Release (Fix Issues)
+
+Any issue identified during the community review and vote should be fixed in this step.
+
+To fully cacel a vote:
+
+* Cancel the current release and verify the version is back to the correct SNAPSHOT:
+
+```
+mvn release:cancel
+```
+
+* Drop the release tag:
+
+```
+git tag -d x.x.x
+git push --delete apache x.x.x
+```
+
+* Drop the staging repository on Nexus ([repository.apache.org](https://repository.apache.org))
+
+
+Verify the version is back to the correct SNAPSHOT.
+
+Code changes should be proposed as standard pull requests and merged.
+
+Once all issues have been resolved, you should go back and build a new release candidate
with 
+these changes.
+
+## Finalize the release
+
+Once the release candidate has been reviewed and approved by the community, the release should
be
+ finalized. This involves the final deployment of the release to the release repositories,

+ merging the website changes, and announce the release.
+ 
+### Deploy artifacts to Maven Central repository
+
+On Nexus, release the staged artifacts to Maven Central repository. In the `Staging Repositories`
+ section, find the relevant release candidate `orgapachecarbondata-XXX` entry and click `Release`.
+
+### Deploy source release to dist.apache.org
+
+Copy the source release from the `dev` repository to `release` repository at `dist.apache.org`

+using Subversion.
+
+### Merge website pull request
+
+Merge the website pull request to list the release created earlier.
+
+### Mark the version as released in Jira
+
+In Jira, inside [version management](https://issues.apache.org/jira/plugins/servlet/project-config/CARBONDATA/versions)
+, hover over the current release and a settings menu will appear. Click `Release`, and select

+today's state.
+
+_Checklist to proceed to the next step:_
+
+1. Maven artifacts released and indexed in the
+ [Maven Central repository](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.carbondata%22)
+2. Source distribution available in the release repository of
+ [dist.apache.org](https://dist.apache.org/repos/dist/release/incubator/carbondata/)
+3. Website pull request to list the release merged
+4. Release version finalized in Jira
+
+## Promote the release
+
+Once the release has been finalized, the last step of the process is to promote the release

+within the project and beyond.
+
+### Apache mailing lists
+
+Announce on the dev@ mailing list that the release has been finished.
+ 
+Announce on the user@ mailing list that the release is available, listing major improvements
and 
+contributions.
+
+While in incubation, announce the release on the Incubator's general@ mailing list.
+
+_Checklist to declare the process completed:_
+
+1. Release announced on the user@ mailing list.
+2. Release announced on the Incubator's general@ mailing list.
+3. Completion declared on the dev@ mailing list.

http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/bc1361dd/src/site/markdown/supported-data-types-in-carbondata.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/supported-data-types-in-carbondata.md b/src/site/markdown/supported-data-types-in-carbondata.md
index d71b59b..8f271e3 100644
--- a/src/site/markdown/supported-data-types-in-carbondata.md
+++ b/src/site/markdown/supported-data-types-in-carbondata.md
@@ -22,17 +22,19 @@
 #### CarbonData supports the following data types:
 
   * Numeric Types
-  * SMALLINT
-  * INT/INTEGER
-  * BIGINT
-  * DOUBLE
-  * DECIMAL
+    * SMALLINT
+    * INT/INTEGER
+    * BIGINT
+    * DOUBLE
+    * DECIMAL
 
   * Date/Time Types
-  * TIMESTAMP
+    * TIMESTAMP
+    * DATE
 
   * String Types
-  * STRING
+    * STRING
+    * CHAR
 
   * Complex Types
     * arrays: ARRAY``<data_type>``



Mime
View raw message