carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipes...@apache.org
Subject [4/7] carbondata git commit: [CARBONDATA-1490] Fixed memory allocation for carbon row
Date Fri, 22 Sep 2017 11:07:11 GMT
[CARBONDATA-1490] Fixed memory allocation for carbon row

While creating carbon row the space for measures is assigned based on outputLength and not
on measureCount which will tend to allocate more space than is actually needed.

outputLength = measureCount + ((noDictCount + complexCount) > 0 ? 1 : 0) + 1.

When the table has 1 complex column then measureCount=0, noDictCount=0, complexCount=1 therefore,
outputLength = 2. Even if there are no measure columns the measure object is created with
2 null values.

This closes #1373


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/d4eabbe5
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/d4eabbe5
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/d4eabbe5

Branch: refs/heads/branch-1.2
Commit: d4eabbe56aa6825d84933ab5a0fc78f26762cea9
Parents: e9c24c5
Author: kunal642 <kunalkapoor642@gmail.com>
Authored: Wed Sep 20 11:02:00 2017 +0530
Committer: manishgupta88 <tomanishgupta18@gmail.com>
Committed: Wed Sep 20 14:41:38 2017 +0530

----------------------------------------------------------------------
 .../newflow/steps/CarbonRowDataWriterProcessorStepImpl.java        | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/d4eabbe5/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
----------------------------------------------------------------------
diff --git a/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
b/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
index 765e0ed..643d2ba 100644
--- a/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
+++ b/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
@@ -254,7 +254,7 @@ public class CarbonRowDataWriterProcessorStepImpl extends AbstractDataLoadProces
       nonDicArray[nonDicIndex++] = (byte[]) row.getObject(dimCount);
     }
 
-    Object[] measures = new Object[outputLength];
+    Object[] measures = new Object[measureCount];
     for (int i = 0; i < this.measureCount; i++) {
       measures[i] = row.getObject(i + this.dimensionWithComplexCount);
     }


Mime
View raw message