spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject spark git commit: [SPARK-25760][DOCS][FOLLOWUP] Add note about AddJar return value change in migration guide
Date Fri, 26 Oct 2018 14:48:20 GMT
Repository: spark
Updated Branches:
  refs/heads/master 33e337c11 -> f1891ff1e

[SPARK-25760][DOCS][FOLLOWUP] Add note about AddJar return value change in migration guide

## What changes were proposed in this pull request?

Add note about AddJar return value change in migration guide

## How was this patch tested?


Closes #22826 from srowen/SPARK-25760.2.

Authored-by: Sean Owen <>
Signed-off-by: Sean Owen <>


Branch: refs/heads/master
Commit: f1891ff1e3f03668ac21b352b009bfea5e3c2b7f
Parents: 33e337c
Author: Sean Owen <>
Authored: Fri Oct 26 09:48:17 2018 -0500
Committer: Sean Owen <>
Committed: Fri Oct 26 09:48:17 2018 -0500

 docs/ | 2 ++
 1 file changed, 2 insertions(+)
diff --git a/docs/ b/docs/
index 38c03d3..c9685b8 100644
--- a/docs/
+++ b/docs/
@@ -15,6 +15,8 @@ displayTitle: Spark SQL Upgrading Guide
   - Since Spark 3.0, the `from_json` functions supports two modes - `PERMISSIVE` and `FAILFAST`.
The modes can be set via the `mode` option. The default mode became `PERMISSIVE`. In previous
versions, behavior of `from_json` did not conform to either `PERMISSIVE` nor `FAILFAST`, especially
in processing of malformed JSON records. For example, the JSON string `{"a" 1}` with the schema
`a INT` is converted to `null` by previous versions but Spark 3.0 converts it to `Row(null)`.
+  - The `ADD JAR` command previously returned a result set with the single value 0. It now
returns an empty result set.
 ## Upgrading From Spark SQL 2.3 to 2.4
   - In Spark version 2.3 and earlier, the second parameter to array_contains function is
implicitly promoted to the element type of first array type parameter. This type promotion
can be lossy and may cause `array_contains` function to return wrong result. This problem
has been addressed in 2.4 by employing a safer type promotion mechanism. This can cause some
change in behavior and are illustrated in the table below.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message