carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-247) Higher MAXCOLUMNS value in load DML options is leading to out of memory error
Date Sat, 17 Sep 2016 20:28:20 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15499635#comment-15499635
] 

ASF GitHub Bot commented on CARBONDATA-247:
-------------------------------------------

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-carbondata/pull/162


> Higher MAXCOLUMNS value in load DML options is leading to out of memory error
> -----------------------------------------------------------------------------
>
>                 Key: CARBONDATA-247
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-247
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: dhatchayani
>            Priority: Minor
>
> When a higher value lets say Integer max value is configured for maxcolumns option in
load DML and executor memory is less, then in that case UnivocityCsvParser throws an out of
memory error when it tries to create an array of size of maxColumns option value.
> java.lang.OutOfMemoryError: Java heap space
> 	at com.univocity.parsers.common.ParserOutput.<init>(ParserOutput.java:86)
> 	at com.univocity.parsers.common.AbstractParser.<init>(AbstractParser.java:66)
> 	at com.univocity.parsers.csv.CsvParser.<init>(CsvParser.java:50)
> 	at org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114)
> 	at org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427)
> 	at org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60)
> 	at org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message