From issues-return-30598-archive-asf-public=cust-asf.ponee.io@carbondata.apache.org  Thu Jan 11 11:02:34 2018
Return-Path: <issues-return-30598-archive-asf-public=cust-asf.ponee.io@carbondata.apache.org>
X-Original-To: archive-asf-public@eu.ponee.io
Delivered-To: archive-asf-public@eu.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by mx-eu-01.ponee.io (Postfix) with ESMTP id ED71D180656
	for <archive-asf-public@eu.ponee.io>; Thu, 11 Jan 2018 11:02:33 +0100 (CET)
Received: by cust-asf.ponee.io (Postfix)
	id DD8A5160C20; Thu, 11 Jan 2018 10:02:33 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 2FF22160C1F
	for <archive-asf-public@cust-asf.ponee.io>; Thu, 11 Jan 2018 11:02:33 +0100 (CET)
Received: (qmail 6341 invoked by uid 500); 11 Jan 2018 10:02:32 -0000
Mailing-List: contact issues-help@carbondata.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@carbondata.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@carbondata.apache.org>
List-Post: <mailto:issues@carbondata.apache.org>
List-Id: <issues.carbondata.apache.org>
Reply-To: dev@carbondata.apache.org
Delivered-To: mailing list issues@carbondata.apache.org
Received: (qmail 6332 invoked by uid 99); 11 Jan 2018 10:02:32 -0000
Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Jan 2018 10:02:32 +0000
Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33)
	id 33885E945F; Thu, 11 Jan 2018 10:02:32 +0000 (UTC)
From: ravipesala <git@git.apache.org>
To: issues@carbondata.apache.org
Reply-To: issues@carbondata.apache.org
References: <git-pr-1782-carbondata@git.apache.org>
In-Reply-To: <git-pr-1782-carbondata@git.apache.org>
Subject: [GitHub] carbondata pull request #1782: [CARBONDATA-2019] Enhancement of merge index ...
Content-Type: text/plain
Message-Id: <20180111100232.33885E945F@git1-us-west.apache.org>
Date: Thu, 11 Jan 2018 10:02:32 +0000 (UTC)

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1782#discussion_r160909350
  
    --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala ---
    @@ -893,33 +893,53 @@ object CommonUtil {
     
       /**
        * Merge the carbonindex files with in the segment to carbonindexmerge file inside same segment
    +   *
    +   * @param sparkContext
    +   * @param segmentIds
    +   * @param tablePath
    +   * @param carbonTable
    +   * @param mergeIndexProperty
    +   * @param readFileFooterFromCarbonDataFile flag to read file footer information from carbondata
    +   *                                         file. This will used in case of upgrade from version
    +   *                                         which do not store the blocklet info to current version
        */
       def mergeIndexFiles(sparkContext: SparkContext,
           segmentIds: Seq[String],
           tablePath: String,
           carbonTable: CarbonTable,
    -      mergeIndexProperty: Boolean): Unit = {
    +      mergeIndexProperty: Boolean,
    +      readFileFooterFromCarbonDataFile: Boolean = false): Unit = {
         if (mergeIndexProperty) {
    -      new CarbonMergeFilesRDD(sparkContext, AbsoluteTableIdentifier.from(tablePath,
    -        carbonTable.getDatabaseName, carbonTable.getTableName).getTablePath,
    -        segmentIds).collect()
    +      new CarbonMergeFilesRDD(
    +        sparkContext,
    +        AbsoluteTableIdentifier
    +          .from(tablePath, carbonTable.getDatabaseName, carbonTable.getTableName).getTablePath,
    +        segmentIds,
    +        readFileFooterFromCarbonDataFile).collect()
         } else {
           try {
             CarbonProperties.getInstance()
               .getProperty(CarbonCommonConstants.CARBON_MERGE_INDEX_IN_SEGMENT).toBoolean
             if (CarbonProperties.getInstance().getProperty(
               CarbonCommonConstants.CARBON_MERGE_INDEX_IN_SEGMENT,
               CarbonCommonConstants.CARBON_MERGE_INDEX_IN_SEGMENT_DEFAULT).toBoolean) {
    -          new CarbonMergeFilesRDD(sparkContext, AbsoluteTableIdentifier.from(tablePath,
    -            carbonTable.getDatabaseName, carbonTable.getTableName).getTablePath,
    -            segmentIds).collect()
    +          new CarbonMergeFilesRDD(
    +            sparkContext,
    +            AbsoluteTableIdentifier
    +              .from(tablePath, carbonTable.getDatabaseName, carbonTable.getTableName).getTablePath,
    --- End diff --
    
    Please use like this 
    ``` 
    new CarbonMergeFilesRDD(
                sparkContext,
                AbsoluteTableIdentifier.from(
                  tablePath, 
                  carbonTable.getDatabaseName, 
                  carbonTable.getTableName).getTablePath,
                segmentIds,
                readFileFooterFromCarbonDataFile).collect()
    ```


---