carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipesala <...@git.apache.org>
Subject [GitHub] carbondata pull request #1876: [CARBONDATA-2093] Use small file feature of g...
Date Tue, 30 Jan 2018 08:43:35 GMT
Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1876#discussion_r164671788
  
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
---
    @@ -588,15 +576,13 @@ case class CarbonLoadDataCommand(
             }
             val len = rowDataTypes.length
             var rdd =
    -          new NewHadoopRDD[NullWritable, StringArrayWritable](
    -            sparkSession.sparkContext,
    -            classOf[CSVInputFormat],
    -            classOf[NullWritable],
    -            classOf[StringArrayWritable],
    -            jobConf).map { case (key, value) =>
    +          DataLoadingUtil.csvFileScanRDD(
    +            sparkSession,
    +            model = carbonLoadModel,
    +            hadoopConf).map { row =>
    --- End diff --
    
    ok


---

Mime
View raw message