If the input to my PLL is a very clean clock pulse with almost no jitter, then even if I raise the bandwidth I'm going to draw in any noise. So shouldn't I increase the bandwidth to increase lock time?
You have that backwards. You must increase BW and noise jitter amplification to capture your worst case error frequency, unless using a Type II mixer which has inherent noise from dead-band when rising edges are locked in phase limited by speed of mixer.
Lock time is proportional to loop rise time and error frequency. Even unfilltered error voltage may still be be too small due to control the VCO error frequency to lock ( or capture)
The XOR mixer helps filter noise is also a nonlinear harmonic mixer and does not increase phase error voltage if the signal is off by 1 Hz or 1MHz ( logic level limits are just for phase error) such that the the PLL can lock onto the wrong frequency such as a harmonic of the input signal.
Using an XOR Type I mixer means the phase error voltage is linear but the frequency error is very nonlinear and when out of phase you have a positive feedback loop which pushes the VCO away. So the capture range is limited by the detector charge pump integrator dynamics which must pull harder than push and this difference gets rapidly smaller with error frequency to filter bandwidth ratio, such that it cannot lock on. Thus the filter bandwidth and loop gain must be wider than the worst case error frequency to capture this ac error signal and pull it in 0 phase error.
the Phase Frequency Type II mixer mitigates this error but also amplifies ( essentially dithers around the deadtime inherent to all digital ph/f mixers ) . But this is often necessary if only using an RC VCO and a large frequency error and may be perfectly acceptable for synthesizing a clock where minor jitter is tolerable but not for an RF Tx that multiplies phase noise with divider ratio .
Using a XTAL VCXO allows a smaller error frequency and thus a lower phase noise to capture with a smaller bandwidth.
Using the VCXO to lock onto a Receive signal and hold then allows the Tx synthesized clock to be stable for more tolerant spread-spectrum modulation bursts but yet may not be adequate still for other types of coherent modulation where phase noise must be extremely small for low phase modulation schemes.