Filtering DataType column with Timestamp

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Filtering DataType column with Timestamp

fmilano
Hi. I have a DateType column and I want to filter all the values greater or
equal than a certain Timestamp. This works, for example
df.col(columnName).geq(value) evaluates to a column with DateTypes greater
or equal than value. Except for one case: if the value of the Timestamp is
initialized to "1/1/2018 00:00:00" it filters the columns that are greater
than "1/1/2018 00:00:00". If some rows are initialized to the exact same
date and time it does not include it in the results, that is, the part of
"equal" is not working. If I change the column type to Timestamp this works
fine.

Is this a bug or is it some known behaviour when comparing DateTypes to
Timestamps?

Thanks!



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]