These files are not necessarily version controlled.
FYI: I'm using git and git gui on windows xp.
git diff works on two unversioned paths, both files and directories (recursive diff). I don't think you can do this from the gui; you'll need to use command-line.
If you want GUI, I suggest installing kdiff3 instead. It can also be used by git as merge-tool than if you set
diff.tool = kdiff3,
diff.guitool = kdiff3 and