For every phenomenon, we have two interpretations: one in the time domain, and one in the frequency domain. For the Doppler spread in the frequency domain we have time selectivity in the time domain. This means that the channel changing, which implies that there is motion either from the transmitter, receiver, or the environment.
So, the channel is time varying not because of the constructive and destructive addition of multipath components at a particular point in space, but because of the changes in the channel due to motions. For example, if you are talking from your mobile phone and you not moving, and the environment surrounding you is static, then the channel here is considered to be time-invariant, because the reflected components are always the same. However, if you are walking or the surrounding environment is not static, then the channel is considered time-varying.
At the end, no, we can not have fading if we don't have reflected components, but we may have distortion due to Doppler shift.
Thanks