carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jack...@apache.org
Subject [14/14] carbondata git commit: [CARBONDATA-1965]removed sort_scope from dynamic configuration in carbondata using set-reset as it is not configured by set
Date Sat, 13 Jan 2018 08:10:43 GMT
[CARBONDATA-1965]removed sort_scope from dynamic configuration in carbondata using set-reset
as it is not configured by set

This closes #1784


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/b8bcb0e0
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/b8bcb0e0
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/b8bcb0e0

Branch: refs/heads/carbonstore
Commit: b8bcb0e0eec85f31b7707600da7057a830d46158
Parents: df27831
Author: vandana <vandana.yadav759@gmail.com>
Authored: Wed Jan 10 15:38:40 2018 +0530
Committer: Jacky Li <jacky.likun@qq.com>
Committed: Sat Jan 13 16:02:38 2018 +0800

----------------------------------------------------------------------
 docs/configuration-parameters.md | 2 --
 1 file changed, 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/b8bcb0e0/docs/configuration-parameters.md
----------------------------------------------------------------------
diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md
index 05aede7..338854f 100644
--- a/docs/configuration-parameters.md
+++ b/docs/configuration-parameters.md
@@ -191,11 +191,9 @@ RESET
 | carbon.options.bad.records.logger.enable | To enable or disable bad record logger.    
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                           |
 | carbon.options.bad.records.action        | This property can have four types of actions
for bad records FORCE, REDIRECT, IGNORE and FAIL. If set to FORCE then it auto-corrects the
data by storing the bad records as NULL. If set to REDIRECT then bad records are written to
the raw CSV instead of being loaded. If set to IGNORE then bad records are neither loaded
nor written to the raw CSV. If set to FAIL then data loading fails if any bad records are
found.                                                                                   
                                                                      |
 | carbon.options.is.empty.data.bad.record  | If false, then empty ("" or '' or ,,) data will
not be considered as bad record and vice versa.                                          
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                       |
-| carbon.options.sort.scope                | This property can have four possible values
BATCH_SORT, LOCAL_SORT, GLOBAL_SORT and NO_SORT. If set to BATCH_SORT, the sorting scope is
smaller and more index tree will be created,thus loading is faster but query maybe slower.
If set to LOCAL_SORT, the sorting scope is bigger and one index tree per data node will be
created, thus loading is slower but query is faster. If set to GLOBAL_SORT, the sorting scope
is bigger and one index tree per task will be created, thus loading is slower but query is
faster. If set to NO_SORT data will be loaded in unsorted manner. |
 | carbon.options.batch.sort.size.inmb      | Size of batch data to keep in memory, as a thumb
rule it supposed to be less than 45% of sort.inmemory.size.inmb otherwise it may spill intermediate
data to disk.                                                                            
                                                                                         
                                                                                         
                                                                                         
                                                            |
 | carbon.options.single.pass               | Single Pass Loading enables single job to finish
data loading with dictionary generation on the fly. It enhances performance in the scenarios
where the subsequent data loading after initial load involves fewer incremental updates on
the dictionary. This option specifies whether to use single pass for loading data or not.
By default this option is set to FALSE.                                                  
                                                                                         
                                                                  |
 | carbon.options.bad.record.path           | Specifies the HDFS path where bad records needs
to be stored.                                                                            
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                       |
-| carbon.options.global.sort.partitions    | The Number of partitions to use when shuffling
data for sort. If user don't configurate or configurate it less than 1, it uses the number
of map tasks as reduce tasks. In general, we recommend 2-3 tasks per CPU core in your cluster.
                                                                                         
                                                                                         
                                                                                         
                                                                  |
 | carbon.custom.block.distribution         | Specifies whether to use the Spark or Carbon
block distribution feature.                                                              
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                          |
 | enable.unsafe.sort                       | Specifies whether to use unsafe sort during
data loading. Unsafe sort reduces the garbage collection during data load operation, resulting
in better performance.                                                                   
                                                                                         
                                                                                         
                                                                                         
                                                                      |
 


Mime
View raw message