@SunnySkyguy Thank you for your suggestion (analyzing each section for self-noise etc.). The problem is it doesn't even work in simulation so i know it won't work on the FPGAs.
I will post my blocks now.
Below is the gardner ted; Trig input comes from the vco at decided sampling points.
Below is the loop filter; kp and ki are easy to fiddle with.
Below is the simulink equivalent of my vco which i implemented in system generator. I cannot post the exact model since its not only my property. This block is followed by a "greater than zero comparator" to output pulses from the incoming vco.
These are the basic elements. Resampling circuit is simply a register enabled by the vco signal.
Below is my reference design from matlab.
I am feeding the gardner ted and resamplers with 2 samples per symbol. I do not employ any matched filtering. The above examples TED input looks similar to my real world signal.
@SunnySkyguy Regarding the capture range since the two radios are fpgas the timing difference comes from the boards osc. difference. There is not much difference. I got away with the timing with simpler digital algorithms before up until i hit the 80Msps (40 at I and 40 at Q arm) with no FEC.
The busses at the output of my DDC are 245MHz. 245/40 ~= 6 samples were fine before but past that previous algorithms ( open-loop algorithms ) start to fail or become too dangerous. I came to the point that before employing higher order modulations to increase data throughput, I have the solve the timing recovery with less samples per symbols.
Should I build a model with more oversampling and lower modulations and start from there? I tried that also to no success but I can go that road again.
Regards.