cmos ring oscillator vcos
Basically, it is difficult to suggest a way to control the output amplitude without knowing the structure of you delay cell in the VCO. Having an amplitude not dependent on the tuning is a good idea since it makes the transfer characteristic of the VCO more linear. Are you using something like Maneatis ring oscillator?
As for converting the sin wave to sqare wave - it is my opinion that you should have the waveforms of the ring oscillator as close to sqare as possible - this means that your cells are fast enough and that all transient processes have more or less settled - hence you are less prone to uncertainties and jitter. If your ring oscillator can not produce something close to sqare wave, think about reducing the amplitude, then it will have enough time to settle. What technology do you use?
Second, you don't take the signal from the ring oscillator directly. You use some sort of a buffer - which will also be responsible for converting your reduced swing to full CMOS output. This is a kind of comparator and it also isolates the ring from the outside world.