Mocking pyspark read writes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Mocking pyspark read writes

Dark Crusader
Hi everyone,

I have a function which reads and writes a parquet file from HDFS. When I'm writing a unit test for this function, I want to mock this read & write.

How do you achieve this? 
Any help would be appreciated. Thank you.

Reply | Threaded
Open this post in threaded view
|

Re: Mocking pyspark read writes

Jörn Franke
Write to a local temp directory via file:// ?

> Am 07.07.2020 um 20:07 schrieb Dark Crusader <[hidden email]>:
>
> 
> Hi everyone,
>
> I have a function which reads and writes a parquet file from HDFS. When I'm writing a unit test for this function, I want to mock this read & write.
>
> How do you achieve this?
> Any help would be appreciated. Thank you.
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]