A serious bug in the fitting of a binary logistic regression.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

A serious bug in the fitting of a binary logistic regression.

Yakov Kerzhner

I have written up a JIRA, and there is a gist attached that has code that reproduces the issue.  This is a fairly serious issue as it probably affects everyone who uses spark to fit binary logistic regressions.
https://issues.apache.org/jira/browse/SPARK-34448
Would be great if someone who understands binary logistic regressions and the implementation in scala to take a look.

Reply | Threaded
Open this post in threaded view
|

Re: A serious bug in the fitting of a binary logistic regression.

srowen
I'll take a look. At a glance - is it converging? might turn down the tolerance to check.
Also what does scikit learn say on the same data? we can continue on the JIRA.

On Mon, Feb 22, 2021 at 5:42 PM Yakov Kerzhner <[hidden email]> wrote:

I have written up a JIRA, and there is a gist attached that has code that reproduces the issue.  This is a fairly serious issue as it probably affects everyone who uses spark to fit binary logistic regressions.
https://issues.apache.org/jira/browse/SPARK-34448
Would be great if someone who understands binary logistic regressions and the implementation in scala to take a look.