My question is can I explicitly assign values to variables that are sitting in GPU memory or do I always have to use cudaMemcpy?
for example can i do this:
int * a;
*a = 5;
int * dev_a;
int a = 5;
cudaMemcpy(dev_a, &a, sizeof(int) , cudaMemcpyHostToDevice);
Simple answer is, that you can not assign values to device memory. This has two reasons. The reason is, that the device memory is not in ram, but physically on the gpu and you have to transfer the values over the pcie bridge.
The more complex answer is, that you can use unified memory since cuda 6, which does what you want to do. Keep in mind, that the transfer does happen in the background and does take time.