carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jackylk <...@git.apache.org>
Subject [GitHub] incubator-carbondata pull request #412: [CARBONDATA-519]Added vector reader ...
Date Mon, 12 Dec 2016 14:55:29 GMT
Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/412#discussion_r91961576
  
    --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonScanRDD.scala
---
    @@ -150,12 +153,25 @@ class CarbonScanRDD[V: ClassTag](
         val attemptContext = new TaskAttemptContextImpl(new Configuration(), attemptId)
         val format = prepareInputFormatForExecutor(attemptContext.getConfiguration)
         val inputSplit = split.asInstanceOf[CarbonSparkPartition].split.value
    -    val reader = format.createRecordReader(inputSplit, attemptContext)
    +    val model = format.getQueryModel(inputSplit, attemptContext)
    +    val reader = {
    +      if (vectorReader) {
    +        val carbonRecordReader = createVectorizedCarbonRecordReader(model)
    +        if (carbonRecordReader == null) {
    +          new CarbonRecordReader(model, format.getReadSupportClass(attemptContext.getConfiguration))
    +        } else {
    +          carbonRecordReader
    +        }
    +      } else {
    +        new CarbonRecordReader(model, format.getReadSupportClass(attemptContext.getConfiguration))
    --- End diff --
    
    should not new CarbonRecordReader directly, can we choose:
    option 1: create two InputFormat, one for batch and another for non-batch
    option 2: one InputFormat, create RecordReader according to configuration.
    I prefer option 2.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message