When the phase shift is close to 180 degrees, you still have a lot of open loop gain. This means that the denominator of G/1+GH can still be written as ~|GH| which would mean that the transfer function can be simplified as ~ 1/H. Of course there can be a lot of exceptions. But in most cases, it means that the closed loop transfer function does not become infinity at a frequency and therefore it is stable.
Only where the loop gain comes close to 1 (or 0dB) and the phase shift is close to 180 degrees, will there be a chance that the closed loop TF goes to infinity and therefore you will have oscillations.