Is the CMFB in the Pre-Amplifier stage of the Comparator, which could be a linear(ish) stage?
A pre-amplifier output which has a properly defined output common mode could be used to drive the final latch/comparator. This could be helpful in reducing the delays since the differential output of the preamplifier will be at 0Vdiff and known Vcm and on application of the inputs, it does not have to swing wildy.
This is my guess. A circuit diagram would be great...!