Registers in x86 PCs are limited to 32 bits, which means the biggest number that can be stored in that register is 2^32, or 4,294,967,296; which means that the CPU can access up to "ADDRESS 4,294,967,296" which is the 4th GB of data.
The problem is that the CPU also access I/O devices (printer, USB, keyboard, mouse, monitor, etc.) by assigning them an address as well. So it reserves half a GB or so of the memory addresses to talk to the I/O devices... So you have 4GB - ~0.5GB -> 3.5 GB (on Windows it is 3.2GB).