carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jack...@apache.org
Subject carbondata git commit: [CARBONDATA-2938][DataMap] Update comment of blockletId in IndexDataMapRebuildRDD
Date Sun, 16 Sep 2018 02:06:51 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master 2fb7dc9a7 -> c65a6527d


[CARBONDATA-2938][DataMap] Update comment of blockletId in IndexDataMapRebuildRDD

This closes #2723


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/c65a6527
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/c65a6527
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/c65a6527

Branch: refs/heads/master
Commit: c65a6527d1818840a958f1b249360d23bab874ef
Parents: 2fb7dc9
Author: Manhua <kevinjmh@qq.com>
Authored: Sat Sep 15 10:30:00 2018 +0800
Committer: Jacky Li <jacky.likun@qq.com>
Committed: Sun Sep 16 10:06:14 2018 +0800

----------------------------------------------------------------------
 .../org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala   | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/c65a6527/integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala
----------------------------------------------------------------------
diff --git a/integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala
b/integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala
index 5e07f1b..0584fb1 100644
--- a/integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala
+++ b/integration/spark2/src/main/scala/org/apache/carbondata/datamap/IndexDataMapRebuildRDD.scala
@@ -356,7 +356,9 @@ class IndexDataMapRebuildRDD[K, V](
         // skip clear datamap and we will do this adter rebuild
         reader.setSkipClearDataMapAtClose(true)
 
-        // Note that blockletId in rowWithPosition does not work properly,
+        // Note that datamap rebuilding is based on query, the blockletId in rowWithPosition
+        // is set to relative number in carbondata file in query process.
+        // In order to get absolute blockletId in shard like the one filled in loading process,
         // here we use another way to generate it.
         var blockletId = 0
         var firstRow = true


Mime
View raw message