spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-17174) Provide support for Timestamp type Column in add_months function to return HH:mm:ss
Date Wed, 24 Aug 2016 01:43:20 GMT

    [ https://issues.apache.org/jira/browse/SPARK-17174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434038#comment-15434038
] 

Apache Spark commented on SPARK-17174:
--------------------------------------

User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/14778

> Provide support for Timestamp type Column in add_months function to return HH:mm:ss
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-17174
>                 URL: https://issues.apache.org/jira/browse/SPARK-17174
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core, SQL
>    Affects Versions: 2.0.0
>            Reporter: Amit Baghel
>            Priority: Minor
>
> add_months function currently supports Date types. If Column is Timestamp type then it
adds month to date but it doesn't return timestamp part (HH:mm:ss). See the code below.
> {code}
> import java.util.Calendar
> val now = Calendar.getInstance().getTime()
> val df = sc.parallelize((0 to 3).map(i => {now.setMonth(i); (i, new java.sql.Timestamp(now.getTime))}).toSeq).toDF("ID",
"DateWithTS")
> df.withColumn("NewDateWithTS", add_months(df("DateWithTS"),1)).show
> {code}
> Above code gives following response. See the HH:mm:ss is missing from NewDateWithTS column.
> {code}
> +---+--------------------+-------------+
> | ID|          DateWithTS|NewDateWithTS|
> +---+--------------------+-------------+
> |  0|2016-01-21 09:38:...|   2016-02-21|
> |  1|2016-02-21 09:38:...|   2016-03-21|
> |  2|2016-03-21 09:38:...|   2016-04-21|
> |  3|2016-04-21 09:38:...|   2016-05-21|
> +---+--------------------+-------------+
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message