PySpark with custom transformer project organization

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

PySpark with custom transformer project organization

femibyte

I have a Pyspark project that requires a custom ML Pipeline Transformer written in Scala. What is the best practice regarding project organization ? Should I include the scala files in the general Python project or should they be in a separate repo ?

Opinions and suggestions welcome.

Sent from my iPhone
---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]