I have seen a few people that have git repos with their dot files. I'm wondering if they just
git add .vimrc
Making a git repository of your home is probably a bad idea (you would be spending more time creating your .gitignore file than on doing what you really want to do).
I suggest using a separate git directory for your dotfiles (eg.
~/git/dotfiles) and them making symlinks to your home (eg.
ln -s ~/git/dotfiles/.vim ~/.vim, etc.).
If you can't be bothered creating symlinks manually each time you want to install your dotfiles somewhere, you can use a script like the following one: https://github.com/sitaktif/dotfiles/blob/master/bin/create-symlinks (it uses https://github.com/sitaktif/dotfiles/blob/master/CONFIG as the configuration).