carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jack...@apache.org
Subject carbondata git commit: [CARBONDATA-2117]Fixed Syncronization issue in CarbonEnv
Date Sat, 03 Feb 2018 09:36:50 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master 6fd778ab1 -> 44e70d08e


[CARBONDATA-2117]Fixed Syncronization issue in CarbonEnv

Problem: When creating multiple session (100) session initialisation is failing with below
error

java.lang.IllegalArgumentException: requirement failed: Config entry enable.unsafe.sort already
registered!

Solution: Currently in CarbonEnv we are updating global configuration(shared) and location
configuration in class level synchronized block. In case of multiple session class level lock
will not work , need to add global level lock so only one thread will update the global configuration

This closes #1908


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/44e70d08
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/44e70d08
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/44e70d08

Branch: refs/heads/master
Commit: 44e70d08e0c73e2c65e9a0d147cbbbe965aaf9f7
Parents: 6fd778a
Author: kumarvishal <kumarvishal.1802@gmail.com>
Authored: Thu Feb 1 23:13:54 2018 +0530
Committer: Jacky Li <jacky.likun@qq.com>
Committed: Sat Feb 3 17:36:40 2018 +0800

----------------------------------------------------------------------
 .../spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala    | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/44e70d08/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
----------------------------------------------------------------------
diff --git a/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala b/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
index 40035ce..6b12008 100644
--- a/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
+++ b/integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
@@ -68,7 +68,8 @@ class CarbonEnv {
 
     // added for handling timeseries function like hour, minute, day , month , year
     sparkSession.udf.register("timeseries", new TimeSeriesFunction)
-    synchronized {
+    // acquiring global level lock so global configuration will be updated by only one thread
+    CarbonEnv.carbonEnvMap.synchronized {
       if (!initialized) {
         // update carbon session parameters , preserve thread parameters
         val currentThreadSesssionInfo = ThreadLocalSessionInfo.getCarbonSessionInfo


Mime
View raw message