spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenchen Fan (Jira)" <>
Subject [jira] [Resolved] (SPARK-35102) Make spark.sql.hive.version meaningful and not deprecated
Date Mon, 19 Apr 2021 14:41:00 GMT


Wenchen Fan resolved SPARK-35102.
    Fix Version/s: 3.2.0
       Resolution: Fixed

Issue resolved by pull request 32200

> Make spark.sql.hive.version meaningful and not deprecated
> ---------------------------------------------------------
>                 Key: SPARK-35102
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.2.0
>            Reporter: Kent Yao
>            Assignee: Kent Yao
>            Priority: Minor
>             Fix For: 3.2.0
> Firstly let's take a look the definition and comment.
> {code:java}
> // A fake config which is only here for backward compatibility reasons. This config has
no effect
> // to Spark, just for reporting the builtin Hive version of Spark to existing applications
> // already rely on this config.
> val FAKE_HIVE_VERSION = buildConf("spark.sql.hive.version")
>   .doc(s"deprecated, please use ${HIVE_METASTORE_VERSION.key} to get the Hive version
in Spark.")
>   .version("1.1.1")
> {code}
> It is used for reporting the built-in Hive version but the current status is unsatisfactory,
as it is could be changed in many ways e.g. --conf/SET syntax.
> It is marked as deprecated but kept a long way until now. I guess it is hard for us to
remove it and not even necessary.
> On second thought, it's actually good for us to keep it to work with the `spark.sql.hive.metastore.version`.
As when `spark.sql.hive.metastore.version` is changed, it could just be used to report the
compiled hive version statically, it's useful when an error occurs in this case. So this parameter
should be fixed to compiled hive version.

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message