kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] shaofengshi commented on a change in pull request #347: KYLIN-3666 HDFS metadata url not be recognized
Date Mon, 19 Nov 2018 14:31:51 GMT
shaofengshi commented on a change in pull request #347: KYLIN-3666 HDFS metadata url not be
recognized
URL: https://github.com/apache/kylin/pull/347#discussion_r234639228
 
 

 ##########
 File path: engine-mr/src/main/java/org/apache/kylin/engine/mr/common/AbstractHadoopJob.java
 ##########
 @@ -494,6 +494,17 @@ public static KylinConfig loadKylinConfigFromHdfs(SerializableConfiguration
conf
     }
 
     public static KylinConfig loadKylinConfigFromHdfs(String uri) {
+        KylinConfig config = loadKylinConfigFromHdfsNotThreadLocal(uri);
+        
+        // This is a bad example where the thread local KylinConfig cannot be auto-closed
due to 
+        // limitation of MR API. It works because MR task runs its own process. Do not copy.
+        @SuppressWarnings("unused")
+        SetAndUnsetThreadLocalConfig shouldAutoClose = KylinConfig.setAndUnsetThreadLocalConfig(config);
+
+        return config;
+    }
+
+    public static KylinConfig loadKylinConfigFromHdfsNotThreadLocal(String uri) {
 
 Review comment:
   The method name is confusing. I think the root cause is the bad sample in AbstractHadoopJob.loadKylinConfigFromHdfs(),
we should fix that.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message