I am wondering how much GPU computing would help me speed up my simulations.
The critical part of my code is matrix multiplication. Basically the code looks like the following python code with matrices of order 1000 and long for loops.
import numpy as np
Msize = 1000
simulationLength = 50
a = np.random.rand(Msize,Msize)
b = np.random.rand(Msize,Msize)
for j in range(simulationLength):
result = np.dot(a,b)
If you use
numpy, you are probably using one of the BLAS libraries as computational backend, such as ATLAS, OpenBLAS, MKL, etc. When you are using the fastest one MKL, you can find a recent performance benchmark here, between a recent Nvidia GPU K40m and Intel Xeon 12-core E5-2697 v2 @ 2.70GHz
where K40m is 6x faster than 12-thread E5-2697. Considering MKL scales well on multi-core CPU. K40m is ~72x faster than 1-thread E5-2697. Please also note 1000-dim is almost the lower bound to fully utilise both the GPU and CPU. Smaller matrix size usually leads to more performance degrade on GPU.
If you are using slower BLAS backend for
numpy, say the GNU-licensed ATLAS. You could then find the comparison between MKL and ATLAS here
where MKL is 2~4x faster than ATLAS.
For Nvidia GPUs, the only widely used backend is CUDA's cuBLAS, so the performance won't change a lot like ATLAS vs. MKL.
As @janbrohl says, data transfer between host RAM and GPU device memory is an important factor that affect the overall performance. Here's a benchmark of the data transfer speed.
Given the matrix size, you can actually calculate out the absolute time for computation and data transfer, respectively. These could help you evaluate the performance better.
To maximise the performance on GPU, you probably need re-design you program to minimise the data transfer, by moving all the computational operations to GPU, rather than matrix multiplication only.