is there a way to create new column with timeuuid using raw spark sql ?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

is there a way to create new column with timeuuid using raw spark sql ?

kant kodali
Hi All,

Is there any way to create a new timeuuid column of a existing dataframe using raw sql? you can assume that there is a timeuuid udf function if that helps.

Thanks!
jgp
Reply | Threaded
Open this post in threaded view
|

Re: is there a way to create new column with timeuuid using raw spark sql ?

jgp
Sure, use withColumn()...

jg


> On Feb 1, 2018, at 05:50, kant kodali <[hidden email]> wrote:
>
> Hi All,
>
> Is there any way to create a new timeuuid column of a existing dataframe using raw sql? you can assume that there is a timeuuid udf function if that helps.
>
> Thanks!


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: is there a way to create new column with timeuuid using raw spark sql ?

kant kodali
Hi,

Are you talking about df.withColumn() ? If so, thats not what I meant. I meant creating a new column using raw sql. otherwords say I dont have a dataframe I only have the view name from df.createOrReplaceView("table") so I can do things like "select * from table" so in a similar fashion I want to see how I can create a new Column using the raw sql. I am looking at this reference https://docs.databricks.com/spark/latest/spark-sql/index.html and I am not seeing a way. 

Thanks!

On Thu, Feb 1, 2018 at 4:01 AM, Jean Georges Perrin <[hidden email]> wrote:
Sure, use withColumn()...

jg


> On Feb 1, 2018, at 05:50, kant kodali <[hidden email]> wrote:
>
> Hi All,
>
> Is there any way to create a new timeuuid column of a existing dataframe using raw sql? you can assume that there is a timeuuid udf function if that helps.
>
> Thanks!


Reply | Threaded
Open this post in threaded view
|

Re: is there a way to create new column with timeuuid using raw spark sql ?

Subhash Sriram
If you have the temp view name (table, for example), couldn't you do something like this?

val dfWithColumn=spark.sql("select *, <your_new_column> as new_column from table")

Thanks,
Subhash

On Thu, Feb 1, 2018 at 11:18 AM, kant kodali <[hidden email]> wrote:
Hi,

Are you talking about df.withColumn() ? If so, thats not what I meant. I meant creating a new column using raw sql. otherwords say I dont have a dataframe I only have the view name from df.createOrReplaceView("table") so I can do things like "select * from table" so in a similar fashion I want to see how I can create a new Column using the raw sql. I am looking at this reference https://docs.databricks.com/spark/latest/spark-sql/index.html and I am not seeing a way. 

Thanks!

On Thu, Feb 1, 2018 at 4:01 AM, Jean Georges Perrin <[hidden email]> wrote:
Sure, use withColumn()...

jg


> On Feb 1, 2018, at 05:50, kant kodali <[hidden email]> wrote:
>
> Hi All,
>
> Is there any way to create a new timeuuid column of a existing dataframe using raw sql? you can assume that there is a timeuuid udf function if that helps.
>
> Thanks!



Reply | Threaded
Open this post in threaded view
|

Re: is there a way to create new column with timeuuid using raw spark sql ?

Liana Napalkova

unsubscribe


DISCLAIMER: Aquest missatge pot contenir informació confidencial. Si vostè no n'és el destinatari, si us plau, esborri'l i faci'ns-ho saber immediatament a la següent adreça: [hidden email] Si el destinatari d'aquest missatge no consent la utilització del correu electrònic via Internet i la gravació de missatges, li preguem que ens ho comuniqui immediatament.

DISCLAIMER: Este mensaje puede contener información confidencial. Si usted no es el destinatario del mensaje, por favor bórrelo y notifíquenoslo inmediatamente a la siguiente dirección: [hidden email] Si el destinatario de este mensaje no consintiera la utilización del correo electrónico vía Internet y la grabación de los mensajes, rogamos lo ponga en nuestro conocimiento de forma inmediata.

DISCLAIMER: Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message you should destroy this message, and notify us immediately to the following address: [hidden email]. If the addressee of this message does not consent to the use of Internet e-mail and message recording, please notify us immediately.