is it ok to make I/O calls in UDF? other words is it a standard practice ?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

is it ok to make I/O calls in UDF? other words is it a standard practice ?

kant kodali
Hi All,

Is it ok to make I/O calls in UDF? other words is it a standard practice?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: is it ok to make I/O calls in UDF? other words is it a standard practice ?

Sathish Kumaran Vairavelu
I have made simple rest call within UDF and it worked but not sure if it can be applied for large datasets but may be for small lookup files. Thanks
On Mon, Apr 23, 2018 at 4:28 PM kant kodali <[hidden email]> wrote:
Hi All,

Is it ok to make I/O calls in UDF? other words is it a standard practice?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: is it ok to make I/O calls in UDF? other words is it a standard practice ?

kant kodali
yes for sure it works. I am just not sure if it is a good idea when getting a stream of messages from kafka?

On Mon, Apr 23, 2018 at 2:54 PM, Sathish Kumaran Vairavelu <[hidden email]> wrote:
I have made simple rest call within UDF and it worked but not sure if it can be applied for large datasets but may be for small lookup files. Thanks

On Mon, Apr 23, 2018 at 4:28 PM kant kodali <[hidden email]> wrote:
Hi All,

Is it ok to make I/O calls in UDF? other words is it a standard practice?

Thanks!

Reply | Threaded
Open this post in threaded view
|

Re: is it ok to make I/O calls in UDF? other words is it a standard practice ?

Jörn Franke
In reply to this post by kant kodali
What is your use case?

> On 23. Apr 2018, at 23:27, kant kodali <[hidden email]> wrote:
>
> Hi All,
>
> Is it ok to make I/O calls in UDF? other words is it a standard practice?
>
> Thanks!

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: is it ok to make I/O calls in UDF? other words is it a standard practice ?

Jungtaek Lim
Another thing you may want to be aware is, if the result is not idempotent, your query result is also not idempotent. For fault-tolerance there's a chance for record (row) to be replayed (recomputed).

-Jungtaek Lim (HeartSaVioR)

2018년 4월 24일 (화) 오후 2:07, Jörn Franke <[hidden email]>님이 작성:
What is your use case?

> On 23. Apr 2018, at 23:27, kant kodali <[hidden email]> wrote:
>
> Hi All,
>
> Is it ok to make I/O calls in UDF? other words is it a standard practice?
>
> Thanks!

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]