Persisting Customized Transformer

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Persisting Customized Transformer

Artemis User
We are trying to create a customized transformer for a ML pipeline and
also want to persist the trained pipeline and retrieve it for
production.  To enable persistency, we will have to implement read/write
functions.  However, this is not feasible in Scala since the read/write
methods are private members of the MLModel class.  This problem was
described in a JIRA ticket https://issues.apache.org/jira/browse/SPARK-17048

Although the ticket suggested some workaround, but only in Java. I was
wondering if anyone has tried in Scala?  The scala work-around suggested
in the ticket didn't work for us.  Does anyone know if this issue has
been resolved in 3.0.1 or the upcoming 3.1?  Any suggestion is highly
appreciated.

-- ND


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]