How to Overwrite a saved PySpark ML Model

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

How to Overwrite a saved PySpark ML Model

Aakash Basu-2
Hi,

I am trying to overwrite a Spark ML Logistic Regression Model, but it isn't working.

Tried:
a) lr_model.write.overwrite().save(input_dict["config"]["save_model_path"])
and
b) lr_model.write.overwrite.save(input_dict["config"]["save_model_path"])

This works (if I do not want to overwrite): lr_model.save(input_dict["config"]["save_model_path"])

Error:
[2019-01-21] [04:57:44] [ERROR]  - AttributeError at line 163 of 'function' object has no attribute 'overwrite': 'function' object has no attribute 'overwrite'
[2019-01-21] [04:57:44] [ERROR]  - 'function' object has no attribute 'overwrite'
[2019-01-21] [04:57:44] [ERROR]  - Traceback (most recent call last):
  File "<string>", line 163, in run
AttributeError: 'function' object has no attribute 'overwrite'

What to do?

Thanks,
Aakash.
Reply | Threaded
Open this post in threaded view
|

Re: How to Overwrite a saved PySpark ML Model

Aakash Basu-2
Hey all,

The message seems to be a Java error message, and not a Python one. So, now I tried by calling the writemethod first:

lr_model.write().overwrite().save(input_dict["config"]["save_model_path"])
It is still running, shall update if it works, otherwise shall need your help.

Thanks,
Aakash.

On Mon, Jan 21, 2019 at 5:14 PM Aakash Basu <[hidden email]> wrote:
Hi,

I am trying to overwrite a Spark ML Logistic Regression Model, but it isn't working.

Tried:
a) lr_model.write.overwrite().save(input_dict["config"]["save_model_path"])
and
b) lr_model.write.overwrite.save(input_dict["config"]["save_model_path"])

This works (if I do not want to overwrite): lr_model.save(input_dict["config"]["save_model_path"])

Error:
[2019-01-21] [04:57:44] [ERROR]  - AttributeError at line 163 of 'function' object has no attribute 'overwrite': 'function' object has no attribute 'overwrite'
[2019-01-21] [04:57:44] [ERROR]  - 'function' object has no attribute 'overwrite'
[2019-01-21] [04:57:44] [ERROR]  - Traceback (most recent call last):
  File "<string>", line 163, in run
AttributeError: 'function' object has no attribute 'overwrite'

What to do?

Thanks,
Aakash.