spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gurwls...@apache.org
Subject spark git commit: [MINOR][DOC] automatic type inference supports also Date and Timestamp
Date Thu, 02 Nov 2017 00:30:25 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-2.2 ab87a92a1 -> c311c5e79


[MINOR][DOC] automatic type inference supports also Date and Timestamp

## What changes were proposed in this pull request?

Easy fix in the documentation, which is reporting that only numeric types and string are supported
in type inference for partition columns, while Date and Timestamp are supported too since
2.1.0, thanks to SPARK-17388.

## How was this patch tested?

n/a

Author: Marco Gaido <mgaido@hortonworks.com>

Closes #19628 from mgaido91/SPARK-22398.

(cherry picked from commit b04eefae49b96e2ef5a8d75334db29ef4e19ce58)
Signed-off-by: hyukjinkwon <gurwls223@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c311c5e7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c311c5e7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c311c5e7

Branch: refs/heads/branch-2.2
Commit: c311c5e7976a60a0f67d913ca10fe72a8559148f
Parents: ab87a92
Author: Marco Gaido <mgaido@hortonworks.com>
Authored: Thu Nov 2 09:30:03 2017 +0900
Committer: hyukjinkwon <gurwls223@gmail.com>
Committed: Thu Nov 2 09:30:20 2017 +0900

----------------------------------------------------------------------
 docs/sql-programming-guide.md | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/c311c5e7/docs/sql-programming-guide.md
----------------------------------------------------------------------
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md
index 9a54adc..8cd4d05 100644
--- a/docs/sql-programming-guide.md
+++ b/docs/sql-programming-guide.md
@@ -778,10 +778,11 @@ root
 {% endhighlight %}
 
 Notice that the data types of the partitioning columns are automatically inferred. Currently,
-numeric data types and string type are supported. Sometimes users may not want to automatically
-infer the data types of the partitioning columns. For these use cases, the automatic type
inference
-can be configured by `spark.sql.sources.partitionColumnTypeInference.enabled`, which is default
to
-`true`. When type inference is disabled, string type will be used for the partitioning columns.
+numeric data types, date, timestamp and string type are supported. Sometimes users may not
want
+to automatically infer the data types of the partitioning columns. For these use cases, the
+automatic type inference can be configured by
+`spark.sql.sources.partitionColumnTypeInference.enabled`, which is default to `true`. When
type
+inference is disabled, string type will be used for the partitioning columns.
 
 Starting from Spark 1.6.0, partition discovery only finds partitions under the given paths
 by default. For the above example, if users pass `path/to/table/gender=male` to either


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message