carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chenliang...@apache.org
Subject carbondata git commit: [CARBONDATA-1880] Documentation for merging small files
Date Sat, 03 Feb 2018 08:06:11 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master 71f8828be -> b48a8c21f


[CARBONDATA-1880] Documentation for merging small files

Documentation for merging small files

This closes #1903


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/b48a8c21
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/b48a8c21
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/b48a8c21

Branch: refs/heads/master
Commit: b48a8c21f75d642c5729bdc3f147a50685447f65
Parents: 71f8828
Author: sgururajshetty <sgururajshetty@gmail.com>
Authored: Wed Jan 31 19:25:16 2018 +0530
Committer: chenliang613 <chenliang613@huawei.com>
Committed: Sat Feb 3 16:05:56 2018 +0800

----------------------------------------------------------------------
 docs/configuration-parameters.md | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/b48a8c21/docs/configuration-parameters.md
----------------------------------------------------------------------
diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md
index b68a2d1..621574d 100644
--- a/docs/configuration-parameters.md
+++ b/docs/configuration-parameters.md
@@ -61,6 +61,7 @@ This section provides the details of all the configurations required for
CarbonD
 | carbon.options.bad.record.path |  | Specifies the HDFS path where bad records are stored.
By default the value is Null. This path must to be configured by the user if bad record logger
is enabled or bad record action redirect. | |
 | carbon.enable.vector.reader | true | This parameter increases the performance of select
queries as it fetch columnar batch of size 4*1024 rows instead of fetching data row by row.
| |
 | carbon.blockletgroup.size.in.mb | 64 MB | The data are read as a group of blocklets which
are called blocklet groups. This parameter specifies the size of the blocklet group. Higher
value results in better sequential IO access.The minimum value is 16MB, any value lesser than
16MB will reset to the default value (64MB). |  |
+| carbon.task.distribution | block | **block**: Setting this value will launch one task per
block. This setting is suggested in case of concurrent queries and queries having big shuffling
scenarios. **custom**: Setting this value will group the blocks and distribute it uniformly
to the available resources in the cluster. This enhances the query performance but not suggested
in case of concurrent queries and queries having big shuffling scenarios. **blocklet**: Setting
this value will launch one task per blocklet. This setting is suggested in case of concurrent
queries and queries having big shuffling scenarios. **merge_small_files**: Setting this value
will merge all the small partitions to a size of (128 MB) during querying. The small partitions
are combined to a map task to reduce the number of read task. This enhances the performance.
| | 
 
 * **Compaction Configuration**
   


Mime
View raw message