carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipes...@apache.org
Subject [29/42] carbondata git commit: condition for single pass
Date Thu, 15 Jun 2017 11:50:34 GMT
condition for single pass


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/bbcc487a
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/bbcc487a
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/bbcc487a

Branch: refs/heads/branch-1.1
Commit: bbcc487a423d1ba42efd1926457fe27d763c00af
Parents: 917152a
Author: sgururajshetty <sgururajshetty@gmail.com>
Authored: Tue May 30 16:42:32 2017 +0530
Committer: ravipesala <ravi.pesala@gmail.com>
Committed: Thu Jun 15 13:17:34 2017 +0530

----------------------------------------------------------------------
 docs/configuration-parameters.md    | 2 +-
 docs/dml-operation-on-carbondata.md | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/bbcc487a/docs/configuration-parameters.md
----------------------------------------------------------------------
diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md
index c63f73d..b71cdbc 100644
--- a/docs/configuration-parameters.md
+++ b/docs/configuration-parameters.md
@@ -130,7 +130,7 @@ This section provides the details of all the configurations required for
CarbonD
   
 | Parameter | Default Value | Description |
 |---------------------------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality
columns of the dictionary code are automatically recognized and these columns will not be
used as global dictionary encoding. If the parameter is false, all dictionary encoding columns
are used as dictionary encoding. The high cardinality column must meet the following requirements:
value of cardinality > configured value of high.cardinality. Equally, the value of cardinality
is higher than the threshold.value of cardinality/ row number x 100 > configured value
of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data
row number is higher than the configured percentage. |
+| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality
columns of the dictionary code are automatically recognized and these columns will not be
used as global dictionary encoding. If the parameter is false, all dictionary encoding columns
are used as dictionary encoding. The high cardinality column must meet the following requirements:
value of cardinality > configured value of high.cardinality. Equally, the value of cardinality
is higher than the threshold.value of cardinality/ row number x 100 > configured value
of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data
row number is higher than the configured percentage. Note: If SINGLE_PASS is used during data
load, then this property will be disabled.|
 | high.cardinality.threshold | 1000000  | It is a threshold to identify high cardinality
of the columns.If the value of columns' cardinality > the configured value, then the columns
are excluded from dictionary encoding. |
 | high.cardinality.row.count.percentage | 80 | Percentage to identify whether column cardinality
is more than configured percent of total row count.Configuration value formula:Value of cardinality/
row number x 100 > configured value of high.cardinality.row.count.percentage. The value
of the parameter must be larger than 0. |
 | carbon.cutOffTimestamp | 1970-01-01 05:30:00 | Sets the start date for calculating the
timestamp. Java counts the number of milliseconds from start of "1970-01-01 00:00:00". This
property is used to customize the start of position. For example "2000-01-01 00:00:00". The
date must be in the form "carbon.timestamp.format". NOTE: The CarbonData supports data store
up to 68 years from the cut-off time defined. For example, if the cut-off time is 1970-01-01
05:30:00, then the data can be stored up to 2038-01-01 05:30:00. |

http://git-wip-us.apache.org/repos/asf/carbondata/blob/bbcc487a/docs/dml-operation-on-carbondata.md
----------------------------------------------------------------------
diff --git a/docs/dml-operation-on-carbondata.md b/docs/dml-operation-on-carbondata.md
index e315468..45f7464 100644
--- a/docs/dml-operation-on-carbondata.md
+++ b/docs/dml-operation-on-carbondata.md
@@ -146,6 +146,9 @@ You can use the following options to load data:
    * If this option is set to TRUE then data loading will take less time.
 
    * If this option is set to some invalid value other than TRUE or FALSE then it uses the
default value.
+   
+   * If this option is set to TRUE, then high.cardinality.identify.enable property will be
disabled during data load.
+   
 ### Example:
 
 ```


Mime
View raw message