Giora Simchoni - 2 months ago 29
R Question

# R: Calculate cosine distance between rows of two sparse matrices

I have two sparse matrices A and B (

`slam::simple_triplet_matrix`
) of the same MxN dimensions, where M = ~100K, N = ~150K.

I wish to calculate the cosine distance between each pair of rows (i.e. row 1 from matrix A and row 1 from matrix B, row 2 from matrix A and row 2 from matrix B, etc.).

I can do this using a for-loop or using
`apply`
function but that's too slow, something like:

``````library(slam)

A <- simple_triplet_matrix(1:3, 1:3, 1:3)
B <- simple_triplet_matrix(1:3, 3:1, 1:3)

cosine <- NULL
for (i in 1:(dim(A)[1])) {
a <- as.vector(A[i,])
b <- as.vector(B[i, ])
cosine[i] <- a %*% b / sqrt(a%*%a * b%*%b)
}
``````

I understand something in this previously asked question might help me, but:

a) This isn't really what I want, I just want M cosine distances for M rows, not all pairwise correlations between rows of a given sparse matrix.

b) I admit to not fully understanding the math behind this 'vectorized' solution so maybe some explanation would come in handy.

Thank you.

EDIT: This is also NOT a duplicate of this question as I'm not just interested in a regular cosine computation on two simple vectors (I clearly know how to do this, see above), I'm interested in a much larger scale situation, specifically involving slam sparse matrices.

According to the documentation, element-by-element (array) multiplication of compatible `simple_triplet_matrices` and `row_sums` of `simple_triplet_matrices` are available. With these operators/functions, the computation is:

``````cosineDist <- function(A, B){
row_sums(A * B) / sqrt(row_sums(A * A) * row_sums(B * B))
}
``````

Notes:

1. `row_sums(A * B)` computes the dot product of each row in `A` and its corresponding row in `B`, which is the numerator term in your `cosine`. The result is a vector (not sparse) whose elements are these dot products for each corresponding row in A and B.
2. `row_sums(A * A)` computes the squared 2-norm of each row in `A`. The result is a vector (not sparse) whose elements are these squared 2-norms for each row in A.
3. Similarly, `row_sums(B * B)` computes the squared 2-norm of each row in `B`. The result is a vector (not sparse) whose elements are these squared 2-norms for each row in B.
4. The rest of the computation operates on these vectors whose elements are for each row of A and/or B.