carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kunalkap...@apache.org
Subject carbondata git commit: [CARBONDATA-2394] Setting segments in thread local space but was not getting reflected in the driver is fixed
Date Mon, 30 Apr 2018 13:42:29 GMT
Repository: carbondata
Updated Branches:
  refs/heads/branch-1.3 2de691833 -> 9d8648ef8


[CARBONDATA-2394] Setting segments in thread local space but was not getting reflected in
the driver is fixed

Problem : from multiple thread if we are setting any property to thread level. It is affecting
other also.
Analysis : information in carbonSessionInfo object is being corrupted inside CarbonEnv. so
if new thread may be impacted by prev one.
Solution : create a new object and in set the user specified values in thread/session params
to new object.

This closes #2223


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/9d8648ef
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/9d8648ef
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/9d8648ef

Branch: refs/heads/branch-1.3
Commit: 9d8648ef889db8bec83e49ccb86186089506589b
Parents: 2de6918
Author: rahulforallp <rahul.kumar@knoldus.in>
Authored: Tue Apr 24 18:08:41 2018 +0530
Committer: kunal642 <kunalkapoor642@gmail.com>
Committed: Mon Apr 30 19:04:13 2018 +0530

----------------------------------------------------------------------
 .../src/main/scala/org/apache/spark/sql/CarbonEnv.scala      | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/9d8648ef/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
----------------------------------------------------------------------
diff --git a/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala b/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
index ed7e83b..21b465d 100644
--- a/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
+++ b/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
@@ -75,10 +75,14 @@ class CarbonEnv {
         // update carbon session parameters , preserve thread parameters
         val currentThreadSesssionInfo = ThreadLocalSessionInfo.getCarbonSessionInfo
         carbonSessionInfo = new CarbonSessionInfo()
+        // We should not corrupt the information in carbonSessionInfo object which is at
the
+        // session level. Instead create a new object and in that set the user specified
values in
+        // thread/session params
+        val threadLevelCarbonSessionInfo = new CarbonSessionInfo()
         if (currentThreadSesssionInfo != null) {
-          carbonSessionInfo.setThreadParams(currentThreadSesssionInfo.getThreadParams)
+          threadLevelCarbonSessionInfo.setThreadParams(currentThreadSesssionInfo.getThreadParams)
         }
-        ThreadLocalSessionInfo.setCarbonSessionInfo(carbonSessionInfo)
+        ThreadLocalSessionInfo.setCarbonSessionInfo(threadLevelCarbonSessionInfo)
         val config = new CarbonSQLConf(sparkSession)
         if (sparkSession.conf.getOption(CarbonCommonConstants.ENABLE_UNSAFE_SORT).isEmpty)
{
           config.addDefaultCarbonParams()


Mime
View raw message