The INL/DNL are affected by the non-linearity of your low pass filter, if it is not good enough. I suggest you test it separately (apply a sinewave and measure the distortion).
You should see if the distortion introduced by the LPF is low enough. Any distortion problem in the filter would worsen the INL that you are measuring. You should evaluate the distortion using a good signal generator (measure its distortion too) and a spectrum analyzer (it makes the FFT inside).