Using cosinSimilarity method for getting pairwise documents similarity

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Using cosinSimilarity method for getting pairwise documents similarity

Soheil Pourbafrani
Hi, 

I got the TF-IDF vector for the documents and store it in an RDD and convert into RowMatrix type:

val mat = new RowMatrix(tweets_tfidf)
Every element of RDD is a sparse Vector related to a document.
The problem is the cosinSimilarity compute the similarity between columns. Is there any way to make it compute the similarity between rows? Or I should transpose the matrix?