If your mains frequency is 50Hz, ripple will be at 100Hz, or one ripple cycle every 10mS.
How can it possibly have a time constant of 1mS and still get rid of the ripple ?
The best you will ever be able to do is average over several cycles.
What you need is a very sharp low pass filter set to something like 20Hz.
That will get rid of all the spikes and ripple, and the dc coming out will follow the average ac amplitude reasonably quickly in either direction.
The sharper the cut off frequency, the higher you can make it, and if your low pass filter has a good phase response without excessive ringing or overshoot it should do everything expected of it fairly well.