Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E8E29200CAF for ; Thu, 8 Jun 2017 05:54:48 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E767F160BE5; Thu, 8 Jun 2017 03:54:48 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3A172160BD0 for ; Thu, 8 Jun 2017 05:54:48 +0200 (CEST) Received: (qmail 88219 invoked by uid 500); 8 Jun 2017 03:54:47 -0000 Mailing-List: contact commits-help@carbondata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.apache.org Delivered-To: mailing list commits@carbondata.apache.org Received: (qmail 88210 invoked by uid 99); 8 Jun 2017 03:54:47 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Jun 2017 03:54:47 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 13361DFE22; Thu, 8 Jun 2017 03:54:47 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: qiangcai@apache.org To: commits@carbondata.apache.org Date: Thu, 08 Jun 2017 03:54:47 -0000 Message-Id: <53f3eaab0b7b4b0c9cb07379653463d5@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/2] carbondata git commit: condition for single pass archived-at: Thu, 08 Jun 2017 03:54:49 -0000 Repository: carbondata Updated Branches: refs/heads/master 3e102dc73 -> da3a68374 condition for single pass Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/5aed309b Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/5aed309b Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/5aed309b Branch: refs/heads/master Commit: 5aed309b5c8b50f72aa96ffb76b46619641815d4 Parents: 3e102dc Author: sgururajshetty Authored: Tue May 30 16:42:32 2017 +0530 Committer: QiangCai Committed: Thu Jun 8 11:49:38 2017 +0800 ---------------------------------------------------------------------- docs/configuration-parameters.md | 2 +- docs/dml-operation-on-carbondata.md | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/5aed309b/docs/configuration-parameters.md ---------------------------------------------------------------------- diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md index c63f73d..b71cdbc 100644 --- a/docs/configuration-parameters.md +++ b/docs/configuration-parameters.md @@ -130,7 +130,7 @@ This section provides the details of all the configurations required for CarbonD | Parameter | Default Value | Description | |---------------------------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality columns of the dictionary code are automatically recognized and these columns will not be used as global dictionary encoding. If the parameter is false, all dictionary encoding columns are used as dictionary encoding. The high cardinality column must meet the following requirements: value of cardinality > configured value of high.cardinality. Equally, the value of cardinality is higher than the threshold.value of cardinality/ row number x 100 > configured value of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data row number is higher than the configured percentage. | +| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality columns of the dictionary code are automatically recognized and these columns will not be used as global dictionary encoding. If the parameter is false, all dictionary encoding columns are used as dictionary encoding. The high cardinality column must meet the following requirements: value of cardinality > configured value of high.cardinality. Equally, the value of cardinality is higher than the threshold.value of cardinality/ row number x 100 > configured value of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data row number is higher than the configured percentage. Note: If SINGLE_PASS is used during data load, then this property will be disabled.| | high.cardinality.threshold | 1000000 | It is a threshold to identify high cardinality of the columns.If the value of columns' cardinality > the configured value, then the columns are excluded from dictionary encoding. | | high.cardinality.row.count.percentage | 80 | Percentage to identify whether column cardinality is more than configured percent of total row count.Configuration value formula:Value of cardinality/ row number x 100 > configured value of high.cardinality.row.count.percentage. The value of the parameter must be larger than 0. | | carbon.cutOffTimestamp | 1970-01-01 05:30:00 | Sets the start date for calculating the timestamp. Java counts the number of milliseconds from start of "1970-01-01 00:00:00". This property is used to customize the start of position. For example "2000-01-01 00:00:00". The date must be in the form "carbon.timestamp.format". NOTE: The CarbonData supports data store up to 68 years from the cut-off time defined. For example, if the cut-off time is 1970-01-01 05:30:00, then the data can be stored up to 2038-01-01 05:30:00. | http://git-wip-us.apache.org/repos/asf/carbondata/blob/5aed309b/docs/dml-operation-on-carbondata.md ---------------------------------------------------------------------- diff --git a/docs/dml-operation-on-carbondata.md b/docs/dml-operation-on-carbondata.md index e315468..45f7464 100644 --- a/docs/dml-operation-on-carbondata.md +++ b/docs/dml-operation-on-carbondata.md @@ -146,6 +146,9 @@ You can use the following options to load data: * If this option is set to TRUE then data loading will take less time. * If this option is set to some invalid value other than TRUE or FALSE then it uses the default value. + + * If this option is set to TRUE, then high.cardinality.identify.enable property will be disabled during data load. + ### Example: ```