Incorrect CAST to TIMESTAMP in Hive compatibility

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Incorrect CAST to TIMESTAMP in Hive compatibility

verbamour
This post has NOT been accepted by the mailing list yet.
Greetings,

I am using Hive compatibility in Spark 2.1.1 and it appears that the CAST string to TIMESTAMP improperly trims the sub-second value. In particular, leading zeros in the decimal portion appear to be dropped.

Steps to reproduce:
1. From `spark-shell` issue: `spark.sql("SELECT CAST('2017-04-05 16:00:48.0297580' AS TIMESTAMP)").show(100, false)`

2. Note erroneous result (i.e. ".0297580" becomes ".29758")
```
+----------------------------------------------+
|CAST(2017-04-05 16:00:48.0297580 AS TIMESTAMP)|
+----------------------------------------------+
|2017-04-05 16:00:48.29758                     |
+----------------------------------------------+
```

I am not currently plugged into the JIRA system for Spark, so if this is truly a bug please bring it to the attention of the appropriate authorities.

Cheers,
 -tom