I'm not an expert in frequency synthesis, but it is possible to FM modulate a simple phased locked loop. The FM would have to be up to 20 KHz just for audio frequencies.
If the loop constant is fairly slow (long time to lock) you will be able to modulate the (vco) frequency without the loop losing lock. Attachment shows a typical design.