carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xubo245 (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-1535) Complex data type ARRAY<STRING> do not support auto merge/compaction
Date Thu, 28 Sep 2017 12:38:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

xubo245 updated CARBONDATA-1535:
--------------------------------
    Description: 
When carbon.enable.auto.load.merge was set as true, the table that has complex data type ARRAY<STRING>
 will happen error when compaction(auto merge).
  
 Complex data type ARRAY<STRING>  doesn't support some operations as follow:

# auto merge
#  sql("alter table array_table compact 'major'")
#  sql("alter table array_table compact 'minor'")


{code:java}
java.lang.IndexOutOfBoundsException: 1
	at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
	at scala.collection.immutable.List.apply(List.scala:84)
	at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
	at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
	at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
	at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)

{code}


  was:
When carbon.enable.auto.load.merge was set as true, the table that has complex data type ARRAY<STRING>
 will happen error when compaction(auto merge).
  
 as follow:

# auto merge
#  sql("alter table array_table compact 'major'")
#  sql("alter table array_table compact 'minor'")


{code:java}
java.lang.IndexOutOfBoundsException: 1
	at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
	at scala.collection.immutable.List.apply(List.scala:84)
	at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
	at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
	at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
	at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)

{code}



> Complex data type ARRAY<STRING>  do not support auto merge/compaction
> ---------------------------------------------------------------------
>
>                 Key: CARBONDATA-1535
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1535
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: xubo245
>   Original Estimate: 720h
>  Remaining Estimate: 720h
>
> When carbon.enable.auto.load.merge was set as true, the table that has complex data type
ARRAY<STRING>  will happen error when compaction(auto merge).
>   
>  Complex data type ARRAY<STRING>  doesn't support some operations as follow:
> # auto merge
> #  sql("alter table array_table compact 'major'")
> #  sql("alter table array_table compact 'minor'")
> {code:java}
> java.lang.IndexOutOfBoundsException: 1
> 	at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
> 	at scala.collection.immutable.List.apply(List.scala:84)
> 	at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
> 	at org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
> 	at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
> 	at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
> 	at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
> 	at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
> 	at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message