The question is not about the maximum heap size on a 32-bit OS, given that 32-bit OSes have a maximum addressable memory size of 4GB, and that the JVM's max heap size depends on how much contiguous free memory can be reserved.
I'm more interested in knowing the maximum (both theoretical and practically achievable) heap size for a 32-bit JVM running in a 64-bit OS. Basically, I'm looking at answers similar to the figures in a related question on SO.
As to why a 32-bit JVM is used instead of a 64-bit one, the reason is not technical but rather administrative/bureaucratic - it is probably too late to install a 64-bit JVM in the production environment.
32-bit JVMs which expect to have a single large chunk of memory cannot use more than 4 Gb (since that is the 32 bit limit which also applies to pointers). This includes Sun and - I'm pretty sure - also IBM implementations. I do not know if e.g. JRockit or others have a large memory option with their 32-bit implementations.
If you expect to be hitting this limit you should strongly consider starting a parallel track validating a 64-bit JVM for your production environment so you have that ready for when the 32-bit environment breaks down. Otherwise you will have to do that work under pressure, which is never nice.
Edit 2014-05-15: Oracle FAQ:
The maximum theoretical heap limit for the 32-bit JVM is 4G. Due to various additional constraints such as available swap, kernel address space usage, memory fragmentation, and VM overhead, in practice the limit can be much lower. On most modern 32-bit Windows systems the maximum heap size will range from 1.4G to 1.6G. On 32-bit Solaris kernels the address space is limited to 2G. On 64-bit operating systems running the 32-bit VM, the max heap size can be higher, approaching 4G on many Solaris systems.