I haven't any experience in ADC design, but now I have the opportunity to learn it because I'm designing a low power 4 bits flash ADC.
It's well known that the performance of the comparator constrains the performance of the entire ADC.
For the comparator, I'm exploring the posibility of use a comparator without a preamplifier because of the power constrain. I'm using a circuit like this
The problem that I see is the strong dependency of the response time of the comparator with the common mode voltage (Vindc) that exhibits a behavior like this.
I think all comparators may have different vindc and delays. And it is possible that the lower comparators can't decide when the ADC works at very high speed if vin is small because of its small vindc.
I believe you can increase the tail current to decrease the delay (what is your clock frequency?) of the output response or you could latch the output of the comparator with a delayed version of the sampling clock, say 160ps later. If neither is good for you, add a preamplifier or change the comparator topology.