spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xwu0226 <...@git.apache.org>
Subject [GitHub] spark pull request: [SPARK-15236][SQL][SPARK SHELL] Add spark-defa...
Date Thu, 26 May 2016 20:05:08 GMT
Github user xwu0226 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/13088#discussion_r64811522
  
    --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/Main.scala ---
    @@ -88,10 +88,22 @@ object Main extends Logging {
         }
     
         val builder = SparkSession.builder.config(conf)
    -    if (SparkSession.hiveClassesArePresent) {
    -      sparkSession = builder.enableHiveSupport().getOrCreate()
    -      logInfo("Created Spark session with Hive support")
    +    if (conf.get("spark.sql.catalogImplementation", "hive").toLowerCase == "hive") {
    +      if (SparkSession.hiveClassesArePresent) {
    +        // In the case that the property is not set at all, builder's config
    +        // does not have this value set to 'hive' yet. The original default
    +        // behavior is that when there are hive classes, we use hive catalog.
    +        sparkSession = builder.enableHiveSupport().getOrCreate()
    +        logInfo("Created Spark session with Hive support")
    +      } else {
    +        // Need to change it back to 'in-memory' if no hive classes are found
    +        // in the case that the property is set to hive in spark-defaults.conf
    +        builder.config("spark.sql.catalogImplementation", "in-memory")
    --- End diff --
    
    Yes. Will use this key.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message