carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From qiang...@apache.org
Subject [1/2] carbondata git commit: condition for single pass
Date Thu, 08 Jun 2017 03:54:47 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master 3e102dc73 -> da3a68374


condition for single pass


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/5aed309b
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/5aed309b
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/5aed309b

Branch: refs/heads/master
Commit: 5aed309b5c8b50f72aa96ffb76b46619641815d4
Parents: 3e102dc
Author: sgururajshetty <sgururajshetty@gmail.com>
Authored: Tue May 30 16:42:32 2017 +0530
Committer: QiangCai <qiangcai@qq.com>
Committed: Thu Jun 8 11:49:38 2017 +0800

----------------------------------------------------------------------
 docs/configuration-parameters.md    | 2 +-
 docs/dml-operation-on-carbondata.md | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/5aed309b/docs/configuration-parameters.md
----------------------------------------------------------------------
diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md
index c63f73d..b71cdbc 100644
--- a/docs/configuration-parameters.md
+++ b/docs/configuration-parameters.md
@@ -130,7 +130,7 @@ This section provides the details of all the configurations required for
CarbonD
   
 | Parameter | Default Value | Description |
 |---------------------------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality
columns of the dictionary code are automatically recognized and these columns will not be
used as global dictionary encoding. If the parameter is false, all dictionary encoding columns
are used as dictionary encoding. The high cardinality column must meet the following requirements:
value of cardinality > configured value of high.cardinality. Equally, the value of cardinality
is higher than the threshold.value of cardinality/ row number x 100 > configured value
of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data
row number is higher than the configured percentage. |
+| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality
columns of the dictionary code are automatically recognized and these columns will not be
used as global dictionary encoding. If the parameter is false, all dictionary encoding columns
are used as dictionary encoding. The high cardinality column must meet the following requirements:
value of cardinality > configured value of high.cardinality. Equally, the value of cardinality
is higher than the threshold.value of cardinality/ row number x 100 > configured value
of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data
row number is higher than the configured percentage. Note: If SINGLE_PASS is used during data
load, then this property will be disabled.|
 | high.cardinality.threshold | 1000000  | It is a threshold to identify high cardinality
of the columns.If the value of columns' cardinality > the configured value, then the columns
are excluded from dictionary encoding. |
 | high.cardinality.row.count.percentage | 80 | Percentage to identify whether column cardinality
is more than configured percent of total row count.Configuration value formula:Value of cardinality/
row number x 100 > configured value of high.cardinality.row.count.percentage. The value
of the parameter must be larger than 0. |
 | carbon.cutOffTimestamp | 1970-01-01 05:30:00 | Sets the start date for calculating the
timestamp. Java counts the number of milliseconds from start of "1970-01-01 00:00:00". This
property is used to customize the start of position. For example "2000-01-01 00:00:00". The
date must be in the form "carbon.timestamp.format". NOTE: The CarbonData supports data store
up to 68 years from the cut-off time defined. For example, if the cut-off time is 1970-01-01
05:30:00, then the data can be stored up to 2038-01-01 05:30:00. |

http://git-wip-us.apache.org/repos/asf/carbondata/blob/5aed309b/docs/dml-operation-on-carbondata.md
----------------------------------------------------------------------
diff --git a/docs/dml-operation-on-carbondata.md b/docs/dml-operation-on-carbondata.md
index e315468..45f7464 100644
--- a/docs/dml-operation-on-carbondata.md
+++ b/docs/dml-operation-on-carbondata.md
@@ -146,6 +146,9 @@ You can use the following options to load data:
    * If this option is set to TRUE then data loading will take less time.
 
    * If this option is set to some invalid value other than TRUE or FALSE then it uses the
default value.
+   
+   * If this option is set to TRUE, then high.cardinality.identify.enable property will be
disabled during data load.
+   
 ### Example:
 
 ```


Mime
View raw message