carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhatchayani (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CARBONDATA-247) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
Date Sat, 17 Sep 2016 05:55:20 GMT
dhatchayani created CARBONDATA-247:
--------------------------------------

             Summary: Higher MAXCOLUMNS value in load DML options is leading to out of memory
error
                 Key: CARBONDATA-247
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-247
             Project: CarbonData
          Issue Type: Bug
            Reporter: dhatchayani
            Priority: Minor


When a higher value lets say Integer max value is configured for maxcolumns option in load
DML and executor memory is less, then in that case UnivocityCsvParser throws an out of memory
error when it tries to create an array of size of maxColumns option value.
java.lang.OutOfMemoryError: Java heap space
	at com.univocity.parsers.common.ParserOutput.<init>(ParserOutput.java:86)
	at com.univocity.parsers.common.AbstractParser.<init>(AbstractParser.java:66)
	at com.univocity.parsers.csv.CsvParser.<init>(CsvParser.java:50)
	at org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114)
	at org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427)
	at org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60)
	at org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message