Mapping words to vector sparkml CountVectorizerModel

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Mapping words to vector sparkml CountVectorizerModel

Sandeep Nemuri
Hi All,

I've used CountVectorizerModel in spark ml and got the td-idf of the words.

Output column of a df looks like:

(63709,[0,1,2,3,6,7,8,10,11,13],[0.6095235999680518,0.9946971867717818,0.5151611294911758,0.4371112749198506,3.4968901993588046,0.06806241719930584,1.1156025996012633,3.0425756717399217,0.3760235829400124])

Wanted to get top n words which are mapped with this ranking.

Any pointers on how to achieve this? 

--
  Regards
  Sandeep Nemuri