Stanford Core NLP implementation in Spark

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Stanford Core NLP implementation in Spark

ravirajan2870
This post has NOT been accepted by the mailing list yet.
Hi, I am trying to implement Databricks stanford corenlp implementation in spark. See https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/1233855/3233914658600709/588180/latest.html. It is taking longer than a non parallel code written in  python code. Any body who has implemented the same, would like to hear your experience with the same library.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Stanford Core NLP implementation in Spark

maziyar
This post has NOT been accepted by the mailing list yet.
Hi,

I am using Stanford Simple API (which is what Databricks CoreNLP is using). I have my own UDF that does pos tagging.

Mine is also very slow compare to Stanford benchmark (10k/s) and local tests. Have you found why this could happen?

Thanks
Loading...