Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 165AA200C3D for ; Tue, 14 Mar 2017 10:06:56 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 15169160B7E; Tue, 14 Mar 2017 09:06:56 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 60C69160B7C for ; Tue, 14 Mar 2017 10:06:55 +0100 (CET) Received: (qmail 46980 invoked by uid 500); 14 Mar 2017 09:06:52 -0000 Mailing-List: contact issues-help@carbondata.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.incubator.apache.org Delivered-To: mailing list issues@carbondata.incubator.apache.org Received: (qmail 46971 invoked by uid 99); 14 Mar 2017 09:06:52 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Mar 2017 09:06:52 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 47E8DCCAF5 for ; Tue, 14 Mar 2017 09:06:52 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.02 X-Spam-Level: X-Spam-Status: No, score=-4.02 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id LtrkuV6z7BIW for ; Tue, 14 Mar 2017 09:06:51 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id BC2975F252 for ; Tue, 14 Mar 2017 09:06:50 +0000 (UTC) Received: (qmail 24364 invoked by uid 99); 14 Mar 2017 09:00:10 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Mar 2017 09:00:10 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 0834BDFC15; Tue, 14 Mar 2017 09:00:08 +0000 (UTC) From: ravipesala To: issues@carbondata.incubator.apache.org Reply-To: issues@carbondata.incubator.apache.org References: In-Reply-To: Subject: [GitHub] incubator-carbondata pull request #620: [CARBONDATA-742] Added batch sort to... Content-Type: text/plain Message-Id: <20170314090009.0834BDFC15@git1-us-west.apache.org> Date: Tue, 14 Mar 2017 09:00:08 +0000 (UTC) archived-at: Tue, 14 Mar 2017 09:06:56 -0000 Github user ravipesala commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/620#discussion_r105854283 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -1149,6 +1149,19 @@ public static final String USE_KETTLE_DEFAULT = "false"; + /** + * Sorts the data in batches and writes the batch data to store with index file. + */ + public static final String LOAD_USE_BATCH_SORT = "carbon.load.use.batch.sort"; + + public static final String LOAD_USE_BATCH_SORT_DEFAULT = "true"; + + /** + * Size of batch data to keep in memory, as a thumb rule it supposed + * to be less than 45% of sort.inmemory.size.inmb otherwise it may spill intermediate data to disk + */ + public static final String LOAD_BATCH_SORT_SIZE_INMB = "carbon.load.batch.sort.size.inmb"; --- End diff -- ok --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastructure@apache.org or file a JIRA ticket with INFRA. ---